Deep Neural Networks often inherit spurious correlations embedded in training data and hence may fail to generalize to unseen domains, which have different distributions from the domain to provide training data. M. Arjovsky et al. (2019) introduced the concept out-of-distribution (o.o.d.) risk, which is the maximum risk among all domains, and formulated the issue caused by spurious correlations as a minimization problem of the o.o.d. risk. Invariant Risk Minimization (IRM) is considered to be a promising approach to minimize the o.o.d. risk: IRM estimates a minimum of the o.o.d. risk by solving a bi-level optimization problem. While IRM has attracted considerable attention with empirical success, it comes with few theoretical guarantees. Especially, a solid theoretical guarantee that the bi-level optimization problem gives the minimum of the o.o.d. risk has not yet been established. Aiming at providing a theoretical justification for IRM, this paper rigorously proves that a solution to the bi-level optimization problem minimizes the o.o.d. risk under certain conditions. The result also provides sufficient conditions on distributions providing training data and on a dimension of feature space for the bi-leveled optimization problem to minimize the o.o.d. risk.

翻译：深度神经网络常继承训练数据中的虚假相关性，导致在分布不同于训练数据域的未见领域上泛化失败。M. Arjovsky等人（2019）提出了分布外风险（即所有域中的最大风险）的概念，并将虚假相关性问题表述为分布外风险的最小化问题。不变风险最小化（Invariant Risk Minimization, IRM）被认为是一种有望最小化分布外风险的方法：IRM通过求解双层优化问题来估计分布外风险的最小值。尽管IRM在经验成功中受到广泛关注，但其理论保证尚不充分。特别是，双层优化问题能给出分布外风险最小值的严格理论保证尚未建立。为对IRM提供理论依据，本文严格证明了在特定条件下，双层优化问题的解能够最小化分布外风险。该结果还为训练数据分布需满足的条件以及特征空间维度需满足的条件提供了充分依据，使得双层优化问题能够实现分布外风险的最小化。