This paper provides data-dependent bounds on the expected error of the Gibbs algorithm in the overparameterized interpolation regime, where low training errors are also obtained for impossible data, such as random labels in classification. The results show that generalization in the low-temperature regime is already signaled by small training errors in the noisier high-temperature regime. The bounds are stable under approximation with Langevin Monte Carlo algorithms. The analysis motivates the design of an algorithm to compute bounds, which on the MNIST and CIFAR-10 datasets yield nontrivial, close predictions on the test error for true labeled data, while maintaining a correct upper bound on the test error for random labels.

翻译：本文针对过参数化插值机制下的吉布斯算法，建立了数据依赖的期望误差界。该机制下即使在不可能数据（如分类任务中的随机标签）上也能获得较低的训练误差。研究结果表明：低噪声高温机制下的小训练误差，已经预示着低温机制中的泛化行为。所得误差界在采用朗之万蒙特卡洛算法进行近似时具有稳定性。该分析启发了一种边界计算算法的设计，在MNIST和CIFAR-10数据集上的实验表明：对于真实标注数据，该算法能给出非平凡且接近实测的测试误差预测；对于随机标签数据，则始终能保持对测试误差的正确上界估计。