Traditional machine learning paradigms are based on the assumption that both training and test data follow the same statistical pattern, which is mathematically referred to as Independent and Identically Distributed ($i.i.d.$). However, in real-world applications, this $i.i.d.$ assumption often fails to hold due to unforeseen distributional shifts, leading to considerable degradation in model performance upon deployment. This observed discrepancy indicates the significance of investigating the Out-of-Distribution (OOD) generalization problem. OOD generalization is an emerging topic of machine learning research that focuses on complex scenarios wherein the distributions of the test data differ from those of the training data. This paper represents the first comprehensive, systematic review of OOD generalization, encompassing a spectrum of aspects from problem definition, methodological development, and evaluation procedures, to the implications and future directions of the field. Our discussion begins with a precise, formal characterization of the OOD generalization problem. Following that, we categorize existing methodologies into three segments: unsupervised representation learning, supervised model learning, and optimization, according to their positions within the overarching learning process. We provide an in-depth discussion on representative methodologies for each category, further elucidating the theoretical links between them. Subsequently, we outline the prevailing benchmark datasets employed in OOD generalization studies. To conclude, we overview the existing body of work in this domain and suggest potential avenues for future research on OOD generalization. A summary of the OOD generalization methodologies surveyed in this paper can be accessed at http://out-of-distribution-generalization.com.

翻译：传统机器学习范式基于训练数据与测试数据遵循相同统计模式的假设，这在数学上被称为独立同分布（$i.i.d.$）。然而在实际应用中，由于不可预见的分布偏移，这一独立同分布假设往往难以成立，导致模型在部署后性能显著下降。这一观测到的差异凸显了研究分布外泛化问题的重要性。分布外泛化是机器学习研究的新兴课题，聚焦于测试数据分布与训练数据分布不同的复杂场景。本文首次对分布外泛化进行系统全面的综述，涵盖从问题定义、方法论发展、评估流程到领域启示与未来方向的多维度内容。我们首先对分布外泛化问题进行精确的形式化描述。随后，根据各方法在整体学习流程中的位置，将现有方法论分为三大类：无监督表示学习、监督模型学习与优化方法。针对每类代表性方法，我们提供了深入讨论，并进一步阐明其间的理论关联。接着，我们概述了分布外泛化研究中常用的基准数据集。最后，总结了该领域的现有研究成果，并提出了分布外泛化未来研究的潜在方向。本文所综述的分布外泛化方法论汇总可通过 http://out-of-distribution-generalization.com 获取。