We derive new bounds for the condition number of kernel matrices, which we then use to enhance existing non-asymptotic test error bounds for kernel ridgeless regression (KRR) in the over-parameterized regime for a fixed input dimension. For kernels with polynomial spectral decay, we recover the bound from previous work; for exponential decay, our bound is non-trivial and novel. Our contribution is two-fold: (i) we rigorously prove the phenomena of tempered overfitting and catastrophic overfitting under the sub-Gaussian design assumption, closing an existing gap in the literature; (ii) we identify that the independence of the features plays an important role in guaranteeing tempered overfitting, raising concerns about approximating KRR generalization using the Gaussian design assumption in previous literature.

翻译：我们推导了核矩阵条件数的新界，进而用于增强固定输入维度下过参数化核无脊回归（KRR）的现有非渐近测试误差界。对于具有多项式谱衰减的核，我们重现了已有工作的界；对于指数衰减核，我们得到了非平凡且新颖的界。我们的贡献有两点：（i）在亚高斯设计假设下严格证明了温和过拟合与灾难性过拟合现象，填补了现有文献的空白；（ii）我们发现特征独立性对保证温和过拟合具有重要作用，这对先前文献中使用高斯设计假设近似KRR泛化性能的做法提出了质疑。