We propose a deep learning-based strategy for the training of the radiotherapy dose calculation using limited data based on two known energy spectra within the general range of radiotherapy linear accelerators. We constructed a dose map using complete photon and electron Monte Carlo (MC) simulations with random rectangular field sizes, iso-centers, and gantry angles for pelvic computed tomog. images with Mohan 4 and 24 MV spectrum photon beams. Two trained models, the virtual dose map-MC (VDM-MC) and improved VDM-MC (IVDM-MC) were tested under Mohan 4, 6, 10, 15, and 24 MV energy conditions in rectangular and intensity-modulated radiation therapy (IMRT) fields. A 3D gamma evaluation assessed the model′s performance. For VD-MC, the 3%/3 mm and 2%/2 mm criteria gamma pass rates were 92.58 ± 0.87% and 85.31 ± 1.56%, resp., using the rectangular field test, whereas they were 97.03 ± 0.63% and 90.97 ± 1.46%, resp., using the IMRT field test. For IVD-MC, the 3%/3 mm and 2%/2 mm criteria gamma pass rates were 97.86 ± 0.35% and 92.98 ± 0.77%, resp., using the rectangular field test, and 98.57 ± 0.14% and 95.11 ± 0.33%, resp., using the IMRT field test. Feasibility of a single model to achieve accurate and rapid MC dose calculations for photons with different energy spectra was preliminarily verified. This data augmentation strategy effectively generalised the scope of the model.