Spurious correlations in the data, where multiple cues are predictive of the target labels, often lead to shortcut learning phenomena, where a model may rely on erroneous, easy-to-learn, cues while ignoring reliable ones. In this work, we propose an ensemble diversification framework exploiting the generation of synthetic counterfactuals using Diffusion Probabilistic Models (DPMs). We discover that DPMs have the inherent capability to represent multiple visual cues independently, even when they are largely correlated in the training data. We leverage this characteristic to encourage model diversity and empirically show the efficacy of the approach with respect to several diversification objectives. We show that diffusion-guided diversification can lead models to avert attention from shortcut cues, achieving ensemble diversity performance comparable to previous methods requiring additional data collection.

翻译：数据中存在的虚假关联（即多个线索共同预测目标标签）常导致捷径学习现象——模型可能依赖错误且易学的线索，而忽略可靠线索。本文提出一种集成多样化框架，利用扩散概率模型生成合成反事实样本。我们发现，即使在训练数据中多个视觉线索高度相关，扩散概率模型仍具备独立表征它们的固有能力。我们利用这一特性促进模型多样性，并通过实验验证了该方法在多种多样化目标下的有效性。结果表明，扩散引导的多样化可引导模型规避对捷径线索的关注，其集成多样性表现可与先前需要额外数据收集的方法相媲美。