Counterfactually-Augmented Data (CAD) -- minimal editing of sentences to flip the corresponding labels -- has the potential to improve the Out-Of-Distribution (OOD) generalization capability of language models, as CAD induces language models to exploit domain-independent causal features and exclude spurious correlations. However, the empirical results of CAD's OOD generalization are not as efficient as anticipated. In this study, we attribute the inefficiency to the myopia phenomenon caused by CAD: language models only focus on causal features that are edited in the augmentation operation and exclude other non-edited causal features. Therefore, the potential of CAD is not fully exploited. To address this issue, we analyze the myopia phenomenon in feature space from the perspective of Fisher's Linear Discriminant, then we introduce two additional constraints based on CAD's structural properties (dataset-level and sentence-level) to help language models extract more complete causal features in CAD, thereby mitigating the myopia phenomenon and improving OOD generalization capability. We evaluate our method on two tasks: Sentiment Analysis and Natural Language Inference, and the experimental results demonstrate that our method could unlock the potential of CAD and improve the OOD generalization performance of language models by 1.0% to 5.9%.

翻译：反事实增强数据（CAD）——通过对句子进行最小化编辑以翻转对应标签——有望提升语言模型的分布外泛化能力，因为CAD能引导语言模型利用与领域无关的因果特征、排除虚假相关性。然而，CAD在分布外泛化中的实证效果并未达到预期。本研究中，我们将这种低效性归因于CAD引发的"近视现象"：语言模型仅关注增强操作中被编辑的因果特征，而忽略其他未被编辑的因果特征，导致CAD的潜力未得到充分利用。为解决此问题，我们从Fisher线性判别角度在特征空间中分析了近视现象，并基于CAD的结构特性（数据集层面和句子层面）引入两个额外约束，帮助语言模型在CAD中提取更完整的因果特征，从而缓解近视现象并提升分布外泛化能力。我们在情感分析和自然语言推理两项任务上评估了该方法，实验结果表明，我们的方法能释放CAD的潜力，使语言模型的分布外泛化性能提升1.0%至5.9%。