Out-of-distribution (OOD) generalization is a challenging machine learning problem yet highly desirable in many high-stake applications. Existing methods suffer from overly pessimistic modeling with low generalization confidence. As generalizing to arbitrary test distributions is impossible, we hypothesize that further structure on the topology of distributions is crucial in developing strong OOD resilience. To this end, we propose topology-aware robust optimization (TRO) that seamlessly integrates distributional topology in a principled optimization framework. More specifically, TRO solves two optimization objectives: (1) Topology Learning which explores data manifold to uncover the distributional topology; (2) Learning on Topology which exploits the topology to constrain robust optimization for tightly-bounded generalization risks. We theoretically demonstrate the effectiveness of our approach and empirically show that it significantly outperforms the state of the arts in a wide range of tasks including classification, regression, and semantic segmentation. Moreover, we empirically find the data-driven distributional topology is consistent with domain knowledge, enhancing the explainability of our approach.

翻译：分布外（OOD）泛化是机器学习中的一项挑战性任务，但在许多高风险应用中极为重要。现有方法存在过度悲观的建模问题，导致泛化置信度较低。由于泛化到任意测试分布是不可能的，我们假设分布拓扑结构的进一步建模对增强OOD鲁棒性至关重要。为此，我们提出拓扑感知鲁棒优化（TRO），该方法将分布拓扑无缝集成到原则性的优化框架中。具体而言，TRO求解两个优化目标：（1）拓扑学习——探索数据流形以揭示分布拓扑结构；（2）拓扑学习——利用该拓扑约束鲁棒优化以实现紧致的泛化风险边界。我们从理论上证明了该方法的有效性，并通过实验表明，在分类、回归和语义分割等广泛任务中，TRO显著优于现有技术。此外，实验发现数据驱动的分布拓扑与领域知识一致，增强了方法的可解释性。