We study the problem of out-of-distribution (o.o.d.) generalization where spurious correlations of attributes vary across training and test domains. This is known as the problem of correlation shift and has posed concerns on the reliability of machine learning. In this work, we introduce the concepts of direct and indirect effects from causal inference to the domain generalization problem. We argue that models that learn direct effects minimize the worst-case risk across correlation-shifted domains. To eliminate the indirect effects, our algorithm consists of two stages: in the first stage, we learn an indirect-effect representation by minimizing the prediction error of domain labels using the representation and the class labels; in the second stage, we remove the indirect effects learned in the first stage by matching each data with another data of similar indirect-effect representation but of different class labels in the training and validation phase. Our approach is shown to be compatible with existing methods and improve the generalization performance of them on correlation-shifted datasets. Experiments on 5 correlation-shifted datasets and the DomainBed benchmark verify the effectiveness of our approach.

翻译：我们研究分布外泛化问题，其中属性的伪相关性在不同训练域和测试域之间发生变化。这被称为相关性偏移问题，并已引发对机器学习可靠性的担忧。在本工作中，我们将因果推断中的直接效应和间接效应概念引入领域泛化问题。我们论证，学习直接效应的模型能最小化跨相关性偏移域的最坏情况风险。为消除间接效应，我们的算法包含两个阶段：在第一阶段，我们通过使用表示和类别标签最小化域标签的预测误差，学习间接效应表示；在第二阶段，我们通过在训练和验证阶段将每个数据与另一个具有相似间接效应表示但类别标签不同的数据进行匹配，移除第一阶段学习到的间接效应。我们的方法被证明与现有方法兼容，并能提升它们在相关性偏移数据集上的泛化性能。在5个相关性偏移数据集和DomainBed基准上的实验验证了我们方法的有效性。