Src homology-2 domain containing protein tyrosine phosphatase-2 (SHP2) is a highly attractive therapeutic target for treating Kirsten rat sarcoma viral oncogene (KRAS) mutant cancers. In this work, a series of guanidine-based SHP2 allosteric inhibitors were discovered via virtual screening and rational structural optimization. Notably, lead compound 23 with potent SHP2 inhibitory activity (IC50 = 17.7 nM) effectively inhibited the proliferation, migration, and invasion of MIA PaCa-2 pancreatic cancer cells. Furthermore, compound 23 featured great in vivo pharmacokinetic properties (AUCpo = 4320 nM·h; F = 66.3%) and exhibited significant antitumor efficacy in the MIA PaCa-2 xenograft mouse model. This demonstrates that compound 23 is a potential lead compound for the development of SHP2 allosteric inhibitors to treat KRAS mutant cancers. Moreover, these guanidine-based scaffolds may provide an opportunity to mitigate the potential safety risks of the alkyl amine motif predominately incorporated in current SHP2 allosteric inhibitors.