β-N-Acetylhexosaminidases have emerged as promising targets for drug and pesticide discovery due to their critical physiological functions in various cellular processes. In particular, human O-GlcNAcase (hOGA) from the glycoside hydrolase family 84 (GH84) has gained significant attention. This enzyme was found to be linked to various diseases such as diabetes, cancer, and Alzheimer's disease (AD). In this study, to develop novel hOGA inhibitors with suitable pharmaceutical properties, virtual screening of the Drugbank database was performed using a docking-based approach targeting hOGA. Chlorhexidine (4, Ki = 4.0 μM) was identified as a potent hOGA inhibitor with excellent selectivity (Ki > 200 μM against human β-N-acetylhexosaminidase B) and subjected to structural modifications and SAR studies. Furthermore, molecular dynamics simulations as well as binding free energy and free energy decomposition calculations were carried out to investigate the basis for the efficiency of potent inhibitors against hOGA. This present work revealed the new application of the disinfectant chlorhexidine and provided useful information for the future design of hOGA inhibitors.