Spurious correlations in the data, where multiple cues are predictive of the target labels, often lead to a phenomenon known as shortcut bias, where a model relies on erroneous, easy-to-learn cues while ignoring reliable ones. In this work, we propose an ensemble diversification framework exploiting Diffusion Probabilistic Models (DPMs) for shortcut bias mitigation. We show that at particular training intervals, DPMs can generate images with novel feature combinations, even when trained on samples displaying correlated input features. We leverage this crucial property to generate synthetic counterfactuals to increase model diversity via ensemble disagreement. We show that DPM-guided diversification is sufficient to remove dependence on primary shortcut cues, without a need for additional supervised signals. We further empirically quantify its efficacy on several diversification objectives, and finally show improved generalization and diversification performance on par with prior work that relies on auxiliary data collection.

翻译：数据中的虚假相关性（即多个线索共同预测目标标签）常导致一种被称为捷径偏差的现象——模型依赖错误且易学的线索，而忽略可靠的线索。本文提出一种利用扩散概率模型（DPMs）的集成多样化框架，用于缓解捷径偏差。我们证明，在特定训练间隔下，即使训练样本呈现相关输入特征，扩散概率模型也能生成具有新颖特征组合的图像。我们利用这一关键特性生成合成反事实样本，通过集成不一致性提升模型多样性。实证表明，扩散概率模型引导的多样化足以消除对主要捷径线索的依赖，且无需额外监督信号。我们进一步在多个多样化目标上量化其效能，最终证明其泛化与多样化性能可与依赖辅助数据收集的先前工作相媲美。