Deep learning models exhibit a preference for statistical fitting over logical reasoning. Spurious correlations might be memorized when there exists statistical bias in training data, which severely limits the model performance especially in small data scenarios. In this work, we introduce Counterfactual Adversarial Training framework (CAT) to tackle the problem from a causality perspective. Particularly, for a specific sample, CAT first generates a counterfactual representation through latent space interpolation in an adversarial manner, and then performs Counterfactual Risk Minimization (CRM) on each original-counterfactual pair to adjust sample-wise loss weight dynamically, which encourages the model to explore the true causal effect. Extensive experiments demonstrate that CAT achieves substantial performance improvement over SOTA across different downstream tasks, including sentence classification, natural language inference and question answering.

翻译：深层学习模式比逻辑推理更倾向于统计适当。当培训数据存在统计偏差,严重限制了模型的模型性能,特别是在小型数据假设情景中,深层学习模式就更倾向于统计适宜。当培训数据存在统计偏差时,可以将纯净的相互关系混为一谈。在这项工作中,我们引入了反事实对立培训框架(CAT),以便从因果关系的角度解决问题。特别是对于具体的样本来说,CAT首先以对抗性的方式通过潜伏空间内插法产生反事实代表,然后对每对原始对子进行反事实风险最小化(CRM ), 以动态调整抽样对准损失重量,这鼓励模型探索真实的因果关系效果。广泛的实验表明,CAT在不同下游任务,包括判决分类、自然语言推论和问题回答中,在SOTA上取得了显著的业绩改进。