Musashino University

Marine Pollution Bulletin increasingly applies machine learning and explainable AI to pollutant and shellfish poisoning risk, exemplified by PCA-based source apportionment and SHAP-based feature attribution. However, linear PCA may misrepresent structure in inherently nonlinear environmental data, and existing studies often treat model-derived feature importances as evidence of true associations without assessing consistency or dose-response relationships. This paper clarifies that supervised models possess two distinct accuracies: prediction and feature importance, and only prediction can be validated against ground truth. Using a Basque coastal dataset (8195 instances, 14 features) with chlorophyll-a as a proxy for paralytic shellfish poisoning risk, we introduce a leave-top1-out procedure to test ranking stability. Random Forest and XGBoost with and without SHAP show pronounced instability, indicating biased, model-dependent importances. In contrast, unsupervised and non-target-prediction methods yield perfectly stable rankings while matching or exceeding supervised performance, supporting routine stability, consistency, dose-response, and linearity checks in environmental ML studies.

Supervised machine learning excels at target prediction but can mischaracterize structure-biodegradability associations when feature importance is treated as ground truth. Using the QSAR Biodegradation dataset (1055 chemicals; 41 descriptors), we compare targeted supervised models (random forest, XGBoost, logistic regression), unsupervised methods (feature agglomeration, highly variable gene selection), and non-targeted supervised approaches (Spearman correlation). We evaluate cross-validated accuracy and ranking stability via a top-10 selection protocol and a leave-top-1-out perturbation. XGBoost attains the highest accuracy (0.8569) yet exhibits ranking instability; random forests are similarly unstable. In contrast, unsupervised and non-targeted supervised methods achieve strong accuracy (≈0.819-0.849) with perfect stability. Results caution against equating high predictive accuracy with reliable feature importance and support stability-aware, label-agnostic selection for interpretable materials science.