Contrastive learning (CL) methods effectively learn data representations in a self-supervision manner, where the encoder contrasts each positive sample over multiple negative samples via a one-vs-many softmax cross-entropy loss. By leveraging large amounts of unlabeled image data, recent CL methods have achieved promising results when pretrained on large-scale datasets, such as ImageNet. However, most of them consider the augmented views from the same instance are positive pairs, while views from other instances are negative ones. Such binary partition insufficiently considers the relation between samples and tends to yield worse performance when generalized on images in the wild. In this paper, to further improve the performance of CL and enhance its robustness on various datasets, {we propose a doubly CL strategy that separately compares positive and negative samples within their own groups, and then proceeds with a contrast between positive and negative groups}. We realize this strategy with contrastive attraction and contrastive repulsion (CACR), which makes the query not only exert a greater force to attract more distant positive samples but also do so to repel closer negative samples. Theoretical analysis reveals that CACR generalizes CL's behavior by positive attraction and negative repulsion, and it further considers the intra-contrastive relation within the positive and negative pairs to narrow the gap between the sampled and true distribution, which is important when datasets are less curated. With our extensive experiments, CACR not only demonstrates good performance on CL benchmarks, but also shows better robustness when generalized on imbalanced image datasets. Code and pre-trained checkpoints are available at https://github.com/JegZheng/CACR-SSL.

翻译：对比学习（CL）方法通过自监督方式有效学习数据表示，其中编码器借助一对多的softmax交叉熵损失函数，将每个正样本与多个负样本进行对比。通过利用大量无标注图像数据，近期对比学习方法在ImageNet等大规模数据集上预训练时取得了显著成果。然而，大多数方法将同一实例的增强视图视为正样本对，而其他实例的视图则被视为负样本对。这种二元划分未能充分考虑样本间关系，在推广到野生图像时往往表现欠佳。本文为进一步提升对比学习性能并增强其在多种数据集上的鲁棒性，提出了一种双对比学习策略，该策略分别比较各自组内的正样本和负样本，然后进行正负组之间的对比。我们通过对比吸引与对比排斥（CACR）实现这一策略，使得查询不仅对距离更远的正样本施加更强的吸引力，也对距离更近的负样本施加更强的排斥力。理论分析表明，CACR通过正吸引和负排斥泛化了对比学习的行为，并进一步考虑了正负样本对内部的对比关系，以缩小采样分布与真实分布之间的差距，这在数据集非精心整理时尤为重要。通过大量实验，CACR不仅在对比学习基准上展现出优异性能，而且在推广到不平衡图像数据集时表现出更强的鲁棒性。代码与预训练检查点可在https://github.com/JegZheng/CACR-SSL获取。