We investigate a strategy for improving the efficiency of contrastive learning of visual representations by leveraging a small amount of supervised information during pre-training. We propose a semi-supervised loss, SuNCEt, based on noise-contrastive estimation and neighbourhood component analysis, that aims to distinguish examples of different classes in addition to the self-supervised instance-wise pretext tasks. On ImageNet, we find that SuNCEt can be used to match the semi-supervised learning accuracy of previous contrastive approaches while using less than half the amount of pre-training and compute. Our main insight is that leveraging even a small amount of labeled data during pre-training, and not only during fine-tuning, provides an important signal that can significantly accelerate contrastive learning of visual representations. Our code is available online at github.com/facebookresearch/suncet.

翻译：我们通过在培训前利用少量监管信息,调查提高视觉表现对比学习效率的战略。我们提议根据噪音多变估计和邻里部分分析,采用半监督损失SNCEt,目的是区分不同类别的例子,除了自我监督的以实例为借口的任务之外,还采用自我监督的借口任务。在图像网上,我们发现SNCEt可以用来匹配以往对比方法的半监督学习准确性,同时使用不到培训前和计算数量的一半。我们的主要见解是,在培训前和微调期间,甚至利用少量的标签数据,都提供了一个重要的信号,可以大大加快对视觉表现的对比学习。我们的代码可以在 Github.com/脸书搜索/Sincent网上查阅。