With an increase in global aging, the number of people affected by cerebrovascular diseases is also increasing, and the incidence of vascular dementia—closely related to cerebrovascular risk—is increasing at an epidemic rate. However, few therapeutic options exist that can markedly improve the cognitive impairment and prognosis of vascular dementia patients. Similarly in Alzheimer’s disease and other neurological disorders, synaptic dysfunction is recognized as the main reason for cognitive decline. Nitric oxide is one of the ubiquitous gaseous cellular messengers involved in multiple physiological and pathological processes of the central nervous system. Recently, nitric oxide has been implicated in regulating synaptic plasticity and plays an important role in the pathogenesis of vascular dementia. This review introduces in detail the emerging role of nitric oxide in physiological and pathological states of vascular dementia and summarizes the diverse effects of nitric oxide on different aspects of synaptic dysfunction, neuroinflammation, oxidative stress, and blood–brain barrier dysfunction that underlie the progress of vascular dementia. Additionally, we propose that targeting the nitric oxide-sGC-cGMP pathway using certain specific approaches may provide a novel therapeutic strategy for vascular dementia.