Self-supervised contrastive learning has solved one of the significant obstacles in deep learning by alleviating the annotation cost. This advantage comes with the price of false negative-pair selection without any label information. Supervised contrastive learning has emerged as an extension of contrastive learning to eliminate this issue. However, aside from accuracy, there is a lack of understanding about the impacts of adversarial training on the representations learned by these learning schemes. In this work, we utilize supervised learning as a baseline to comprehensively study the robustness of contrastive and supervised contrastive learning under different adversarial training scenarios. Then, we begin by looking at how adversarial training affects the learned representations in hidden layers, discovering more redundant representations between layers of the model. Our results on CIFAR-10 and CIFAR-100 image classification benchmarks demonstrate that this redundancy is highly reduced by adversarial fine-tuning applied to the contrastive learning scheme, leading to more robust representations. However, adversarial fine-tuning is not very effective for supervised contrastive learning and supervised learning schemes. Our code is released at https://github.com/softsys4ai/CL-Robustness.

翻译：自监督对比学习通过减轻标注成本解决了深度学习中的重大障碍，但这一优势伴随着无标签信息下错误负样本对选择的问题。监督对比学习作为对比学习的扩展应运而生，以消除该问题。然而，除了准确率之外，人们对对抗训练对这些学习方案所学表示的影响仍缺乏理解。本文以监督学习为基线，全面研究了对比学习和监督对比学习在不同对抗训练场景下的鲁棒性。我们首先探究对抗训练如何影响隐藏层中学到的表示，发现模型各层之间存在更多冗余表示。我们在CIFAR-10和CIFAR-100图像分类基准上的结果表明，通过对对比学习方案应用对抗微调，这种冗余性被大幅降低，从而产生更鲁棒的表示。然而，对抗微调对监督对比学习和监督学习方案的效果并不显著。我们的代码已发布在 https://github.com/softsys4ai/CL-Robustness。