Adversarial contrastive learning (ACL), without requiring labels, incorporates adversarial data with standard contrastive learning (SCL) and outputs a robust representation which is generalizable and resistant to adversarial attacks and common corruptions. The style-independence property of representations has been validated to be beneficial in improving robustness transferability. Standard invariant regularization (SIR) has been proposed to make the learned representations via SCL to be independent of the style factors. However, how to equip robust representations learned via ACL with the style-independence property is still unclear so far. To this end, we leverage the technique of causal reasoning to propose an adversarial invariant regularization (AIR) that enforces robust representations learned via ACL to be style-independent. Then, we enhance ACL using invariant regularization (IR), which is a weighted sum of SIR and AIR. Theoretically, we show that AIR implicitly encourages the prediction of adversarial data and consistency between adversarial and natural data to be independent of data augmentations. We also theoretically demonstrate that the style-independence property of robust representation learned via ACL still holds in downstream tasks, providing generalization guarantees. Empirically, our comprehensive experimental results corroborate that IR can significantly improve the performance of ACL and its variants on various datasets.

翻译：对抗对比学习（ACL）无需标签，将对抗数据与标准对比学习（SCL）相结合，输出一种具有泛化能力且能抵抗对抗攻击和常见污染的鲁棒表征。表征的样式无关性已被证实有利于提升鲁棒迁移性。标准不变正则化（SIR）已被提出，以使通过SCL学习的表征独立于样式因素。然而，如何使通过ACL学习的鲁棒表征具备样式无关性目前仍不清楚。为此，我们利用因果推理技术提出了一种对抗不变正则化（AIR），强制通过ACL学习的鲁棒表征具有样式无关性。然后，我们使用不变正则化（IR）增强ACL，该正则化是SIR和AIR的加权和。理论上，我们证明AIR隐式地鼓励对抗数据的预测以及对抗数据与自然数据之间的一致性独立于数据增强。我们还从理论上证明，通过ACL学习的鲁棒表征的样式无关性在下游任务中仍然成立，从而提供了泛化保证。实验上，我们的综合实验结果证实，IR可以显著提升ACL及其变体在各种数据集上的性能。