We study the cost of overfitting in noisy kernel ridge regression (KRR), which we define as the ratio between the test error of the interpolating ridgeless model and the test error of the optimally-tuned model. We take an "agnostic" view in the following sense: we consider the cost as a function of sample size for any target function, even if the sample size is not large enough for consistency or the target is outside the RKHS. We analyze the cost of overfitting under a Gaussian universality ansatz using recently derived (non-rigorous) risk estimates in terms of the task eigenstructure. Our analysis provides a more refined characterization of benign, tempered and catastrophic overfitting (qv Mallinar et al. 2022).

翻译：我们研究了噪声核岭回归中过拟合的代价，将其定义为插值无岭模型测试误差与最优调参模型测试误差之比。我们采用"不可知"视角：即考虑任意目标函数下代价随样本量的变化，即使样本量不足以达到一致性或目标函数不属于再生核希尔伯特空间。基于高斯普适性假设，我们利用最近推导的（非严格）任务特征结构风险估计，分析了过拟合代价。我们的研究为良性、温和与灾难性过拟合（参见Mallinar等人，2022）提供了更精细的表征。