Approximate Leave-One-Out Cross-Validation (ALO-CV) is a method that has been proposed to estimate the generalization error of a regularized estimator in the high-dimensional regime where dimension and sample size are of the same order, the so-called ``proportional regime''. A new analysis is developed to derive the consistency of ALO-CV for non-differentiable regularizers under Gaussian covariates and strong convexity. Using a conditioning argument, the difference between the ALO-CV weights and their counterparts in mean-field inference is shown to be small. Combined with upper bounds between the mean-field inference estimate and the leave-one-out quantity, this provides a proof that ALO-CV approximates the leave-one-out quantity up to negligible error terms. Linear models with square loss, robust linear regression and single-index models are explicitly treated.

翻译：近似留一交叉验证（ALO-CV）是一种在高维场景下估计正则化估计器泛化误差的方法，其中维度与样本量处于相同数量级，即所谓的“比例极限”。本文提出了一种新的分析框架，用于证明在协变量服从高斯分布且损失函数强凸的条件下，ALO-CV对于不可微正则化器的一致性。通过条件化论证，我们证明了ALO-CV权重与平均场推断中对应权重之间的差异是可忽略的。结合平均场推断估计量与留一量之间的上界分析，这为ALO-CV能以可忽略误差逼近留一量提供了严格证明。本文具体处理了平方损失线性模型、鲁棒线性回归以及单指标模型。