Dataset bias, i.e., the over-reliance on dataset-specific literal heuristics, is getting increasing attention for its detrimental effect on the generalization ability of NLU models. Existing works focus on eliminating dataset bias by down-weighting problematic data in the training process, which induce the omission of valid feature information while mitigating bias. In this work, We analyze the causes of dataset bias from the perspective of causal inference and propose CausalAPM, a generalizable literal disentangling framework to ameliorate the bias problem from feature granularity. The proposed approach projects literal and semantic information into independent feature subspaces, and constrains the involvement of literal information in subsequent predictions. Extensive experiments on three NLP benchmarks (MNLI, FEVER, and QQP) demonstrate that our proposed framework significantly improves the OOD generalization performance while maintaining ID performance.

翻译：数据集偏差，即模型过度依赖数据集特有的字面启发式特征，因其对自然语言理解模型泛化能力的有害影响而日益受到关注。现有研究主要通过降低训练过程中问题数据的权重来消除数据集偏差，但这在缓解偏差的同时也导致了有效特征信息的遗漏。本文从因果推断视角分析数据集偏差的成因，提出CausalAPM——一种从特征粒度改善偏差问题的通用化字面解耦框架。该方法将字面信息与语义信息投影至独立的特征子空间，并约束字面信息对后续预测的参与程度。在三个自然语言处理基准数据集（MNLI、FEVER和QQP）上的大量实验表明，本框架在保持域内性能的同时显著提升了域外泛化能力。