Ovarian cancer, often referred to as the 'silent killer,' is a significant contributor to mortality rates. Emerging evidence implicates Nanog as a potential therapeutic target in ovarian cancer. Amcasertib (BBI-503) is an orally administered primary class stemness kinase inhibitor that effectively targets NANOG and various cancer stem cell pathways by specifically inhibiting serine-threonine stemness kinases. This study aimed to evaluate the antineoplastic effects of Nanog inhibition, a critical transcription factor associated with pluripotency and its role in ovarian cancer tumorigenesis, using the novel therapeutic agent Amcasertib in ovarian cancer cells characterized by distinct genetic profiles. The cytotoxicity of Amcasertib was assessed in both ovarian cancer and cancer stem cell models utilizing the Xelligence-RTCA system. The impact of the determined IC50 dose on apoptosis, invasion, migration, epithelial-mesenchymal transition (EMT), cell cycle progression, colony formation, and spheroid growth was evaluated using appropriate analytical techniques. Our findings revealed that Amcasertib exhibited significant antiproliferative effects and induced apoptosis in ovarian cancer and cancer stem cells. Moreover, Amcasertib caused G1 phase arrest and impeded colony formation in MDAH-2774 cells. Additionally, Amcasertib effectively inhibited spheroid growth in OVCAR-3 and OCSC cells. Notably, it demonstrated the ability to suppress invasion and migration in MDAH-2774 and OCSC cells. Furthermore, the suppression of Nanog-mediated stem cell-like features by Amcasertib was particularly pronounced in ER-negative ovarian cancer and cancer stem cells, highlighting its high anticancer efficacy in this subgroup. These results suggest that Amcasertib holds promise as a potential standalone or combination therapy agent for the treatment of ER-negative ovarian cancer.