Adversarial contrastive learning (ACL) is a technique that enhances standard contrastive learning (SCL) by incorporating adversarial data to learn a robust representation that can withstand adversarial attacks and common corruptions without requiring costly annotations. To improve transferability, the existing work introduced the standard invariant regularization (SIR) to impose style-independence property to SCL, which can exempt the impact of nuisance style factors in the standard representation. However, it is unclear how the style-independence property benefits ACL-learned robust representations. In this paper, we leverage the technique of causal reasoning to interpret the ACL and propose adversarial invariant regularization (AIR) to enforce independence from style factors. We regulate the ACL using both SIR and AIR to output the robust representation. Theoretically, we show that AIR implicitly encourages the representational distance between different views of natural data and their adversarial variants to be independent of style factors. Empirically, our experimental results show that invariant regularization significantly improves the performance of state-of-the-art ACL methods in terms of both standard generalization and robustness on downstream tasks. To the best of our knowledge, we are the first to apply causal reasoning to interpret ACL and develop AIR for enhancing ACL-learned robust representations. Our source code is at https://github.com/GodXuxilie/Enhancing_ACL_via_AIR.

翻译：对抗对比学习（ACL）是一种通过整合对抗样本来增强标准对比学习（SCL）的技术，旨在无需昂贵标注的前提下学习能抵御对抗攻击和常见污染的鲁棒表征。为提升可迁移性，现有工作引入标准不变正则化（SIR）赋予SCL风格独立性，从而消除标准表征中干扰性风格因素的影响。然而，风格独立性如何有益于ACL学习的鲁棒表征尚不明确。本文利用因果推理技术解释ACL，并提出对抗不变正则化（AIR）以实现风格因素的独立性约束。我们同时采用SIR和AIR调控ACL，输出鲁棒表征。理论上，我们证明AIR隐式促进自然数据不同视角及其对抗变体之间的表征距离独立于风格因素。实验结果表明，不变正则化显著提升了当前最优ACL方法在下游任务中的标准泛化性能与鲁棒性。据我们所知，这是首次将因果推理应用于解释ACL并开发AIR以增强ACL学习的鲁棒表征。源代码见https://github.com/GodXuxilie/Enhancing_ACL_via_AIR。