Many datasets are biased, namely they contain easy-to-learn features that are highly correlated with the target class only in the dataset but not in the true underlying distribution of the data. For this reason, learning unbiased models from biased data has become a very relevant research topic in the last years. In this work, we tackle the problem of learning representations that are robust to biases. We first present a margin-based theoretical framework that allows us to clarify why recent contrastive losses (InfoNCE, SupCon, etc.) can fail when dealing with biased data. Based on that, we derive a novel formulation of the supervised contrastive loss (epsilon-SupInfoNCE), providing more accurate control of the minimal distance between positive and negative samples. Furthermore, thanks to our theoretical framework, we also propose FairKL, a new debiasing regularization loss, that works well even with extremely biased data. We validate the proposed losses on standard vision datasets including CIFAR10, CIFAR100, and ImageNet, and we assess the debiasing capability of FairKL with epsilon-SupInfoNCE, reaching state-of-the-art performance on a number of biased datasets, including real instances of biases in the wild.

翻译：许多数据集存在偏差，即它们包含与目标类别高度相关的易学特征，但这些特征仅在数据集中成立，而并非数据真实分布中的情况。因此，从有偏数据中学习无偏模型已成为近年来一个重要的研究课题。本文致力于解决如何学习对偏差具有鲁棒性的表征问题。我们首先提出了一个基于间隔的理论框架，用以阐明为何近期提出的对比损失（如InfoNCE、SupCon等）在处理有偏数据时可能失效。基于该框架，我们推导出一种新的监督对比损失公式（epsilon-SupInfoNCE），能够更精确地控制正负样本之间的最小距离。此外，借助我们的理论框架，我们还提出了FairKL——一种新的去偏正则化损失函数，即使在极端有偏数据下也能表现良好。我们在包括CIFAR10、CIFAR100和ImageNet在内的标准视觉数据集上验证了所提出的损失函数，并评估了FairKL与epsilon-SupInfoNCE结合的去偏能力，在多个有偏数据集（包括真实世界中的偏差实例）上达到了最先进的性能。