Contrastive learning is one of the fastest growing research areas in machine learning due to its ability to learn useful representations without labeled data. However, contrastive learning is susceptible to shortcuts - i.e., it may learn shortcut features irrelevant to the task of interest, and discard relevant information. Past work has addressed this limitation via handcrafted data augmentations that eliminate the shortcut. But, manually crafted augmentations do not work across all datasets and tasks. Further, data augmentations fail in addressing shortcuts in multi-attribute classification when one attribute acts as a shortcut around other attributes. In this paper, we analyze the objective function of contrastive learning and formally prove that it is vulnerable to shortcuts. We then present reconstructive contrastive learning (RCL), a framework for learning unsupervised representations that are robust to shortcuts. The key idea is to force the learned representation to reconstruct the input, which naturally counters potential shortcuts. Extensive experiments verify that RCL is highly robust to shortcuts and outperforms state-of-the-art contrastive learning methods on a variety of datasets and tasks.

翻译：反向学习是机器学习中增长最快的研究领域之一,因为它能够学习有用且无标签的数据。 但是,对比学习很容易被捷径所利用, 也就是说, 它可能会学习与感兴趣的任务无关的捷径特征, 并丢弃相关信息。 过去的工作已经通过手工制作的数据扩增解决了这一限制, 从而消除了快捷键。 但是, 手工制作的增强在全部数据集和任务中并不起作用。 此外, 当一个属性作为其他属性的捷径时, 数据扩增无法解决多属性分类中的捷径。 在本文中, 我们分析对比学习的客观功能, 并正式证明它易受到捷径的影响。 我们随后展示了重建式对比学习( RCL ), 这是一种学习不受监督的表达框架, 用来学习对快捷键非常有效的快捷键。 关键的想法是迫使学习的表达方式来重建输入, 这自然相对潜在的捷径。 广泛的实验证实 RCL 在各种数据集和任务上对于快捷键和超常规的对比学习方法非常强大。