Deep Neural Networks often inherit spurious correlations embedded in training data and hence may fail to generalize to unseen domains, which have different distributions from the domain to provide training data. M. Arjovsky et al. (2019) introduced the concept out-of-distribution (o.o.d.) risk, which is the maximum risk among all domains, and formulated the issue caused by spurious correlations as a minimization problem of the o.o.d. risk. Invariant Risk Minimization (IRM) is considered to be a promising approach to minimize the o.o.d. risk: IRM estimates a minimum of the o.o.d. risk by solving a bi-level optimization problem. While IRM has attracted considerable attention with empirical success, it comes with few theoretical guarantees. Especially, a solid theoretical guarantee that the bi-level optimization problem gives the minimum of the o.o.d. risk has not yet been established. Aiming at providing a theoretical justification for IRM, this paper rigorously proves that a solution to the bi-level optimization problem minimizes the o.o.d. risk under certain conditions. The result also provides sufficient conditions on distributions providing training data and on a dimension of feature space for the bi-leveled optimization problem to minimize the o.o.d. risk.

翻译：深度神经网络常继承训练数据中的伪相关性，因此在面对与训练数据分布不同的未见领域时可能泛化失败。M. Arjovsky等人（2019）提出了分布外风险的概念，即所有领域中的最大风险，并将伪相关性引发的问题形式化为分布外风险的最小化问题。不变风险最小化被认为是最小化分布外风险的有效方法：该方法通过求解双层优化问题来估计分布外风险的最小值。尽管IRM凭借实证成功受到广泛关注，但其理论保证尚不完善。特别是，关于双层优化问题能否给出分布外风险最小值的严格理论保证尚未建立。为给IRM提供理论依据，本文严格证明了在一定条件下，双层优化问题的解确实能最小化分布外风险。该结果同时给出了训练数据分布与特征空间维度的充分条件，以确保双层优化问题能够实现分布外风险的最小化。