Cytosolic carboxypeptidases (CCPs) comprise a unique subfamily of M14 carboxypeptidases and are erasers of the reversible protein posttranslational modification- polyglutamylation. Potent inhibitors for CCPs may serve as leading compounds targeting imbalanced polyglutamylation. However, no efficient CCP inhibitor has yet been reported. Here, we showed that 2-phosphonomethylpentanedioic acid (2-PMPA), a potent inhibitor of the distant M28 family member glutamate carboxypeptidase II (GCPII), rather than the typical M14 inhibitor 2-benzylsuccinic acid, could efficiently inhibit CCP activities. 2-PMPA inhibited the recombinant Nna1 (a.k.a. CCP1) for hydrolyzing a synthetic peptide in a mixed manner, with Ki and Ki' being 0.11 μM and 0.24 μM respectively. It inhibited Nna1 for deglutamylating tubulin, the best-known polyglutamylated protein, with an IC50 of 0.21 mM. Homology modeling predicted that the R-form of 2-PMPA is more favorable to bind Nna1, unlike that GCPII prefers to S-form. This work for the first time identified a potent inhibitor for CCP family.