Recent works have shown that self-supervised learning can achieve remarkable robustness when integrated with adversarial training (AT). However, the robustness gap between supervised AT (sup-AT) and self-supervised AT (self-AT) remains significant. Motivated by this observation, we revisit existing self-AT methods and discover an inherent dilemma that affects self-AT robustness: either strong or weak data augmentations are harmful to self-AT, and a medium strength is insufficient to bridge the gap. To resolve this dilemma, we propose a simple remedy named DYNACL (Dynamic Adversarial Contrastive Learning). In particular, we propose an augmentation schedule that gradually anneals from a strong augmentation to a weak one to benefit from both extreme cases. Besides, we adopt a fast post-processing stage for adapting it to downstream tasks. Through extensive experiments, we show that DYNACL can improve state-of-the-art self-AT robustness by 8.84% under Auto-Attack on the CIFAR-10 dataset, and can even outperform vanilla supervised adversarial training for the first time. Our code is available at \url{https://github.com/PKU-ML/DYNACL}.

翻译：近期研究表明，当自监督学习与对抗训练（AT）结合时，能够实现显著的鲁棒性。然而，监督式AT（sup-AT）与自监督式AT（self-AT）之间的鲁棒性差距仍然很大。受此观察启发，我们重新审视了现有的自监督AT方法，并发现了一个影响自监督AT鲁棒性的内在困境：无论是强数据增强还是弱数据增强都会损害自监督AT，而中等强度的增强又不足以弥合这一差距。为解决这一困境，我们提出了一种简单的方法，名为DYNACL（动态对抗性对比学习）。具体而言，我们提出了一种从强增强逐步退火到弱增强的增强调度策略，以充分利用两种极端情况的优势。此外，我们采用了一个快速后处理阶段，以适应下游任务。通过大量实验，我们证明在CIFAR-10数据集上，DYNACL在Auto-Attack攻击下能将自监督AT的最优鲁棒性提升8.84%，并且首次能够超越原始的监督式对抗训练。我们的代码可在\url{https://github.com/PKU-ML/DYNACL}获取。