Simplicity bias is the concerning tendency of deep networks to over-depend on simple, weakly predictive features, to the exclusion of stronger, more complex features. This causes biased, incorrect model predictions in many real-world applications, exacerbated by incomplete training data containing spurious feature-label correlations. We propose a direct, interventional method for addressing simplicity bias in DNNs, which we call the feature sieve. We aim to automatically identify and suppress easily-computable spurious features in lower layers of the network, thereby allowing the higher network levels to extract and utilize richer, more meaningful representations. We provide concrete evidence of this differential suppression & enhancement of relevant features on both controlled datasets and real-world images, and report substantial gains on many real-world debiasing benchmarks (11.4% relative gain on Imagenet-A; 3.2% on BAR, etc). Crucially, we outperform many baselines that incorporate knowledge about known spurious or biased attributes, despite our method not using any such information. We believe that our feature sieve work opens up exciting new research directions in automated adversarial feature extraction & representation learning for deep networks.

翻译：简单性偏差是深度网络的一个令人担忧的倾向，即过度依赖简单但预测能力较弱的特征，而排斥更强且更复杂的特征。这在许多实际应用中导致模型预测出现偏差和错误，且因包含虚假特征-标签相关性的不完整训练数据而加剧。我们提出一种直接且干预性的方法来解决深度神经网络中的简单性偏差，称之为特征筛。该方法旨在自动识别并抑制网络较低层次中易于计算的虚假特征，从而使较高网络层次能够提取并利用更丰富、更有意义的表示。我们在受控数据集和真实世界图像上提供了这种差异性抑制与增强相关特征的具体证据，并在多个现实世界的去偏基准测试中报告了显著性能提升（在ImageNet-A上相对提升11.4%；在BAR上提升3.2%等）。关键在于，尽管我们的方法未使用任何关于已知虚假或偏置属性的先验信息，但其性能仍超越了众多融入此类知识的基线方法。我们相信，特征筛工作为深度网络的自动化对抗性特征提取与表示学习开辟了令人兴奋的新研究方向。