Alzheimer's disease (AD), multifactorial disease, is recognized as one of the most common forms of dementia, and the efficacy of anti-AD drugs is limited clinically. Up-regulated glutaminyl cyclase (QC) and glycogen synthase kinase-3β (GSK-3β) have been identified as two critical elements involved in AD recently. Here, a series of novel chemicals containing maleimide and imidazole motif were designed and synthesized as dual inhibitors targeting QC and GSK-3β. Based on primary screening, compound 2 (2.26 μM), 5 (2.37 μM), 8 (1.34 μM), 21 (2.44 μM), 25 (0.36 μM), 27 (1.76 μM), 28 (1.04 μM), 33 (2.08 μM) and 34 (2.33 μM) exhibited notable human QC (hQC) inhibitory potency, while compound 1 (0.014 μM), 7 (0.04 μM), 8 (0.057 μM), 19 (0.034 μM), 24 (0.014 μM), 32 (0.032 μM), 38 (0.051 μM), 39 (0.044 μM), 44 (0.048 μM), 47 (0.011 μM), 49 (0.021 μM) and so on showed remarkable GSK-3β inhibitory activities. And as expected, these chemicals possessed significant inhibitory potency on both hQC and GSK-3β, such as compound 1 (2.80 and 0.014 μM), 8 (1.34 and 0.057 μM), 25 (0.36 and 0.15 μM), 27 (1.76 and 0.069 μM), 28 (1.04 and 0.090 μM), 33 (2.08 and 0.19 μM), 34 (2.33 and 0.11 μM), 35 (2.55 and 0.14 μM), 36 (2.34 and 0.11 μM), etc. Subsequent in vivo studies demonstrated that compound 8 attenuated cognitive deficits and decreased the anxiety-like behavior in 3 × Tg-AD mice. The treatment decreased both pE-Aβ and Aβ accumulation by inhibiting the activity of QC, and decreased the hyperphosphorylation of Tau by reducing the levels of GSK-3β in the brains of AD mice. Results obtained in this research suggested that these novel compounds could be supposed as potential anti-AD agents targeting QC and GSK-3β.