Neural networks often learn spurious correlations when exposed to biased training data, leading to poor performance on out-of-distribution data. A biased dataset can be divided, according to biased features, into bias-aligned samples (i.e., with biased features) and bias-conflicting samples (i.e., without biased features). Recent debiasing works typically assume that no bias label is available during the training phase, as obtaining such information is challenging and labor-intensive. Following this unsupervised assumption, existing methods usually train two models: a biased model specialized to learn biased features and a target model that uses information from the biased model for debiasing. This paper first presents experimental analyses revealing that the existing biased models overfit to bias-conflicting samples in the training data, which negatively impacts the debiasing performance of the target models. To address this issue, we propose a straightforward and effective method called Echoes, which trains a biased model and a target model with a different strategy. We construct an "echo chamber" environment by reducing the weights of samples which are misclassified by the biased model, to ensure the biased model fully learns the biased features without overfitting to the bias-conflicting samples. The biased model then assigns lower weights on the bias-conflicting samples. Subsequently, we use the inverse of the sample weights of the biased model for training the target model. Experiments show that our approach achieves superior debiasing results compared to the existing baselines on both synthetic and real-world datasets. Our code is available at https://github.com/isruihu/Echoes.

翻译：神经网络在接触有偏训练数据时，常会学习到虚假相关性，导致在分布外数据上表现不佳。根据偏差特征，有偏数据集可分为偏差对齐样本（即包含偏差特征）和偏差冲突样本（即不含偏差特征）。近年来的去偏方法通常假设训练阶段无法获取偏差标签，因为获取此类信息既困难又耗费人力。遵循这一无监督假设，现有方法通常训练两个模型：一个专门学习偏差特征的有偏模型，以及一个利用有偏模型信息进行去偏的目标模型。本文首先通过实验分析揭示：现有有偏模型对训练数据中的偏差冲突样本存在过拟合，这会对目标模型的去偏性能产生负面影响。为解决此问题，我们提出了一种简单有效的方法Echoes，采用差异化策略训练有偏模型和目标模型。我们通过降低被有偏模型误分类样本的权重，构建"回声室"环境，确保有偏模型充分学习偏差特征而不过拟合偏差冲突样本。随后，有偏模型为偏差冲突样本分配更低权重，我们利用有偏模型样本权重的倒数来训练目标模型。实验表明，在合成数据集和真实数据集上，我们的方法均取得了优于现有基线的去偏效果。我们的代码已开源至https://github.com/isruihu/Echoes。