Astragalus membranaceus polysaccharides (APS) possess significant biological activities, such as anti-tumor, antiviral, and immunomodulatory activities. However, there is still a lack of research on the structure-activity relationship of APS. In this paper, two carbohydrate-active enzymes from Bacteroides in living organisms were used to prepare degradation products. The degradation products were divided into APS-A1, APS-G1, APS-G2, and APS-G3 according to molecular weight. Structural analysis showed that all degradation products had an α-1,4-linked glucose backbone, but APS-A1 and APS-G3 also had branched chains of α-1,6-linked galactose or arabinogalacto-oligosaccharide. In vitro, immunomodulatory activity evaluation results indicated that APS-A1 and APS-G3 had better immunomodulatory activity, while the immunomodulatory activities of APS-G1 and APS-G2 were comparatively weaker. Molecular interaction detection showed that APS-A1 and APS-G3 could bind to toll-like receptors-4 (TLR-4) with a binding constant of 4.6 × 10-5 and 9.4 × 10-6, respectively, while APS-G1 and APS-G2 failed to bind to TLR-4. Therefore, the branched chains of galactose or arabinogalacto-oligosaccharide played a crucial role in the immunomodulatory activity of APS.