The effect of the novel IQP-AE-103 (proprietary combination of dehydrated okra powder and inulin) on body weight reduction and the association with changes in microbiota composition were investigated in a double-blind, randomized, placebo-controlled trial. A total of seventy-two overweight or moderately obese subjects with a body mass index of ≥25 and <35 kg/m2 were randomly allocated to receive IQP-AE-103 or placebo; each group received two IQP-AE-103 or placebo capsules three times daily, respectively. Body weight, body fat, waist circumference, and hip circumference were measured, and fecal samples were collected at baseline and after 12 weeks of intervention. Using 16S rRNA gene sequencing on the fecal samples, the microbiota dissimilarity, diversity, and differences in relative abundance between or within groups were analyzed. At the end of the study, body weight was significantly reduced in the IQP-AE-103 group compared with the placebo group, 5.16 ± 2.39 kg vs. 0.97 ± 2.09 kg (p < 0.001). Subjects from the IQP-AE-103 group who achieved a reduction of ≥5% of total body weight from baseline (hereafter referred to as 5% responders or IQP5) had a mean body weight reduction of 6.74 ± 1.94 kg, significantly greater than the placebo group (p < 0.001). Using Lefse and statistical analysis, subjects in the IQP-AE-103 group had a significantly lower relative abundance of Firmicutes than the placebo group (p < 0.05) after 12 weeks of intervention. The 5% responders from the IQP-AE-103 group had a remarkable 4.6-fold higher relative abundance of Akkermansia muciniphila than the placebo group (p < 0.05). As the significant differences between groups were only observed post-intervention, the overall differences in microbiota profile suggest that the weight loss in overweight and moderately obese subjects who consumed IQP-AE-103 for 12 weeks is accompanied by a positive change in microbiota composition. These changes might be linked to the beneficial effects of microbiome modulations in alleviating obesity and metabolic syndrome. To the best of our knowledge, we are the first to report over-the-counter (OTC) supplementation that results in both significant changes in weight and favorable shifts on the subject microbiota profile. The trial is registered under ClinicalTrials.gov Identifier no. NCT03058367.