Powdery mildew is one of the most severe diseases affecting wheat yield and quality and is caused by Blumeria graminis f. sp. tritici (Bgt). Host resistance is the preferred strategy to prevent this disease. However, the narrow genetic basis of common wheat has increased the demand for diversified germplasm resources against powdery mildew. Wheat relatives, especially the secondary gene pool of common wheat, are important gene donors in the genetic improvement of common wheat because of its abundant genetic variation and close kinship with wheat. In this study, a series of 137 wheat relatives, including 53 Triticum monococcum L. (2n = 2x = 14, AA), 6 T. urartu Thumanjan ex Gandilyan (2n = 2x = 14, AA), 9 T. timopheevii Zhuk. (2n = 4x = 28, AAGG), 66 T. aestivum subsp. spelta (2n = 6x = 42, AABBDD), and 3 Aegilops speltoides (2n = 2x = 14, SS) were systematically evaluated for their powdery mildew resistance and composition of Pm genes. Out of 137 (60.58%) accessions, 83 were resistant to Bgt isolate E09 at the seedling stage, and 116 of 137 (84.67%) wheat relatives were resistant to the mixture of Bgt isolates at the adult stage. This indicates that these accessions show a high level of resistance to powdery mildew. Some 31 markers for 23 known Pm genes were used to test these 137 accessions, and, in the results, only Pm2, Pm4, Pm6, Pm58, and Pm68 were detected. Among them, three Pm4 alleles (Pm4a, Pm4b, and Pm4f) were identified in 4 T. subsp. spelta accessions. q-RT PCR further confirmed that Pm4 alleles played a role in disease resistance in these four accessions. The phylogenetic tree showed that the kinship of Pm4 was close to Pm24 and Sr62. This study not only provides reference information and valuable germplasm resources for breeding new wheat varieties with disease resistance but also lays a foundation for enriching the genetic basis of wheat resistance to powdery mildew.