To address the problem of NLP classifiers learning spurious correlations between training features and target labels, a common approach is to make the model's predictions invariant to these features. However, this can be counter-productive when the features have a non-zero causal effect on the target label and thus are important for prediction. Therefore, using methods from the causal inference literature, we propose an algorithm to regularize the learnt effect of the features on the model's prediction to the estimated effect of feature on label. This results in an automated augmentation method that leverages the estimated effect of a feature to appropriately change the labels for new augmented inputs. On toxicity and IMDB review datasets, the proposed algorithm minimises spurious correlations and improves the minority group (i.e., samples breaking spurious correlations) accuracy, while also improving the total accuracy compared to standard training.

翻译：为解决自然语言处理分类器学习训练特征与目标标签之间虚假相关性的问题，常见方法是使模型预测对这些特征保持不变。然而，当特征对目标标签具有非零因果效应因而对预测至关重要时，此类方法可能适得其反。因此，我们借鉴因果推断文献中的方法，提出一种算法，通过正则化特征对模型预测的学习效应，使其逼近特征对标签的估计效应。这形成了一种自动化增强方法：利用特征的估计效应，为新的增强输入合理调整标签。在毒性评论和IMDB评论数据集上，所提算法能最小化虚假相关性，提升少数群体（即打破虚假相关性的样本）的准确率，同时相比标准训练也提高了总体准确率。