We derive new bounds for the condition number of kernel matrices, which we then use to enhance existing non-asymptotic test error bounds for kernel ridgeless regression in the over-parameterized regime for a fixed input dimension. For kernels with polynomial spectral decay, we recover the bound from previous work; for exponential decay, our bound is non-trivial and novel. Our conclusion on overfitting is two-fold: (i) kernel regressors whose eigenspectrum decays polynomially must generalize well, even in the presence of noisy labeled training data; these models exhibit so-called tempered overfitting; (ii) if the eigenspectrum of any kernel ridge regressor decays exponentially, then it generalizes poorly, i.e., it exhibits catastrophic overfitting. This adds to the available characterization of kernel ridge regressors exhibiting benign overfitting as the extremal case where the eigenspectrum of the kernel decays sub-polynomially. Our analysis combines new random matrix theory (RMT) techniques with recent tools in the kernel ridge regression (KRR) literature.

翻译：我们推导了核矩阵的条件数的新边界，并利用这些边界改进了固定输入维度下过参数化无正则化核岭回归的现有非渐近测试误差界。对于多项式谱衰减的核函数，我们恢复了先前工作的边界；对于指数衰减的核函数，我们的边界是非平凡的且具有创新性。关于过拟合问题，我们得到两方面结论：(i) 特征值谱呈多项式衰减的核回归器即使在训练数据含噪声标签时也能良好泛化，这类模型表现出所谓的温和过拟合；(ii) 若任何核岭回归器的特征值谱呈指数衰减，则其泛化性能较差，即表现出灾难性过拟合。这补充了现有关于核岭回归器良性过拟合的刻画——该情形对应于特征值谱呈次多项式衰减的极端情况。我们的分析结合了新的随机矩阵理论方法与核岭回归文献中的最新工具。