BackgroundOverexpression of monopolar spindle 1 (MPS1) and histone deacetylase 8 (HDAC8) is associated with the proliferation of liver cancer cells, so simultaneous inhibition of both MPS1 and HDAC8 could offer a promising therapeutic approach for the treatment of liver cancer. Dual-targeted MPS1/HDAC8 inhibitors have not been reported.MethodsA combined approach of pharmacophore modeling and molecular docking was used to identify potent dual-target inhibitors of MPS1 and HDAC8. Enzyme inhibition assays were performed to evaluate the optimal compound with the strongest inhibitory activity against MPS1 and HDAC8. The selectivity of MPH-5 for MPS1 and HDAC8 was assessed on a panel of 68 kinases and other histone deacetylases. Subsequently, molecular dynamics (MD) simulation verified the binding stability of the optimal compound to MPS1 and HDAC8. Ultimately, in vitro cellular assays and in vivo antitumor assays evaluated the antitumor efficacy of the most promising compound for the treatment of hepatocellular carcinoma.ResultsSix dual-target compounds (MPHs 1-6) of both MPS1 and HDAC8 were identified from the database using a combined virtual screening protocol. Notably, MPH-5 showed nanomolar inhibitory effect on both MPS1 (IC50 = 4.52 ± 0.21 nM) and HDAC8 (IC50 = 6.07 ± 0.37 nM). MD simulation indicated that MPH-5 stably binds to both MPS1 and HDAC8. Importantly, cellular assays revealed that MPH-5 exhibited significant antiproliferative activity against human liver cancer cells, especially HepG2 cells. Moreover, MPH-5 exhibited low toxicity and high efficacy against tumor cells, and it overcomes drug resistance to some extent. In addition, MPH-5 may exert its antitumor effects by downregulating MPS1-driven phosphorylation of histone H3 and upregulating HDAC8-mediated K62 acetylation of PKM2. Furthermore, MPH-5 showed potent inhibition of HepG2 xenograft tumor growth in mice with no apparent toxicity and presented favorable pharmacokinetics.ConclusionThe study suggests that MPH-5 is a potent, selective, high-efficacy, and low-toxicity antitumor candidate for the treatment of hepatocellular carcinoma.