Recent works have shown that self-supervised learning can achieve remarkable robustness when integrated with adversarial training (AT). However, the robustness gap between supervised AT (sup-AT) and self-supervised AT (self-AT) remains significant. Motivated by this observation, we revisit existing self-AT methods and discover an inherent dilemma that affects self-AT robustness: either strong or weak data augmentations are harmful to self-AT, and a medium strength is insufficient to bridge the gap. To resolve this dilemma, we propose a simple remedy named DYNACL (Dynamic Adversarial Contrastive Learning). In particular, we propose an augmentation schedule that gradually anneals from a strong augmentation to a weak one to benefit from both extreme cases. Besides, we adopt a fast post-processing stage for adapting it to downstream tasks. Through extensive experiments, we show that DYNACL can improve state-of-the-art self-AT robustness by 8.84% under Auto-Attack on the CIFAR-10 dataset, and can even outperform vanilla supervised adversarial training for the first time. Our code is available at \url{https://github.com/PKU-ML/DYNACL}.

翻译：近期研究表明，自监督学习在与对抗训练（AT）结合时能够实现显著的鲁棒性。然而，有监督对抗训练（sup-AT）与自监督对抗训练（self-AT）之间的鲁棒性差距依然显著。受此观察启发，我们重新审视了现有自监督对抗训练方法，并发现了一个影响自监督对抗训练鲁棒性的内在困境：无论是强数据增强还是弱数据增强都会对自监督对抗训练造成损害，而中等强度的增强又不足以弥合这一差距。为解决这一困境，我们提出了一种简单补救措施——DYNACL（动态对抗性对比学习）。具体而言，我们设计了一种增强调度策略，从强增强逐渐退火至弱增强，从而兼顾两种极端情况下的优势。此外，我们引入了一个快速后处理阶段，以使其适应下游任务。通过大量实验表明，在CIFAR-10数据集上，DYNACL可在Auto-Attack攻击下将自监督对抗训练的鲁棒性提升8.84%，并且首次超越了普通有监督对抗训练的性能。我们的代码已开源至\url{https://github.com/PKU-ML/DYNACL}。