This paper investigates the efficiency of the K-fold cross-validation (CV) procedure and a debiased version thereof as a means of estimating the generalization risk of a learning algorithm. We work under the general assumption of uniform algorithmic stability. We show that the K-fold risk estimate may not be consistent under such general stability assumptions, by constructing non vanishing lower bounds on the error in realistic contexts such as regularized empirical risk minimisation and stochastic gradient descent. We thus advocate the use of a debiased version of the K-fold and prove an error bound with exponential tail decay regarding this version. Our result is applicable to the large class of uniformly stable algorithms, contrarily to earlier works focusing on specific tasks such as density estimation. We illustrate the relevance of the debiased K-fold CV on a simple model selection problem and demonstrate empirically the usefulness of the promoted approach on real world classification and regression datasets.

翻译：本文研究了K折交叉验证（CV）过程及其去偏版本作为估计学习算法泛化风险方法的效率。我们在算法一致稳定性的通用假设下开展工作。通过构建正则化经验风险最小化和随机梯度下降等现实情境下误差的非消失下界，我们证明在如此一般的稳定性假设下，K折风险估计可能并不一致。因此，我们主张采用K折交叉验证的去偏版本，并证明了该版本的误差指数衰减界。与早期聚焦于密度估计等特定任务的研究不同，我们的结果适用于一大类一致稳定算法。我们在一个简单模型选择问题上阐明了去偏K折CV的相关性，并通过真实世界分类与回归数据集实验验证了所提方法的实用性。