Contrastive learning (CL) is a self-supervised training paradigm that allows us to extract meaningful features without any label information. A typical CL framework is divided into two phases, where it first tries to learn the features from unlabelled data, and then uses those features to train a linear classifier with the labeled data. While a fair amount of existing theoretical works have analyzed how the unsupervised loss in the first phase can support the supervised loss in the second phase, none has examined the connection between the unsupervised loss and the robust supervised loss, which can shed light on how to construct an effective unsupervised loss for the first phase of CL. To fill this gap, our work develops rigorous theories to dissect and identify which components in the unsupervised loss can help improve the robust supervised loss and conduct proper experiments to verify our findings.

翻译：对比学习（CL）是一种自监督训练范式，能够在不依赖任何标签信息的情况下提取有意义的特征。典型的CL框架分为两个阶段：首先从无标签数据中学习特征，然后利用这些特征和标签数据训练线性分类器。尽管现有大量理论工作分析了第一阶段的无监督损失如何支撑第二阶段的监督损失，但尚未有研究探讨无监督损失与鲁棒监督损失之间的联系——这种联系能够揭示如何为CL的第一阶段构建有效的无监督损失。为填补这一空白，我们的工作建立了严格的理论框架，剖析并识别无监督损失中哪些成分有助于改进鲁棒监督损失，同时开展相应实验验证我们的发现。