Gaussian process (GP) regression is a fundamental tool in Bayesian statistics. It is also known as kriging and is the Bayesian counterpart to the frequentist kernel ridge regression. Most of the theoretical work on GP regression has focused on a large-$n$ asymptotics, i.e. as the amount of data increases. Fixed-sample analysis is much more difficult outside of simple cases, such as locations on a regular grid. In this work we perform a fixed-sample analysis that was first studied in the context of approximation theory by Driscoll & Fornberg (2002), called the ``flat limit''. In flat-limit asymptotics, the goal is to characterise kernel methods as the length-scale of the kernel function tends to infinity, so that kernels appear flat over the range of the data. Surprisingly, this limit is well-defined, and displays interesting behaviour: Driscoll & Fornberg showed that radial basis interpolation converges in the flat limit to polynomial interpolation, if the kernel is Gaussian. Subsequent work showed that this holds true in the multivariate setting as well, but that kernels other than the Gaussian may have (polyharmonic) splines as the limit interpolant. Leveraging recent results on the spectral behaviour of kernel matrices in the flat limit, we study the flat limit of Gaussian process regression. Results show that Gaussian process regression tends in the flat limit to (multivariate) polynomial regression, or (polyharmonic) spline regression, depending on the kernel. Importantly, this holds for both the predictive mean and the predictive variance, so that the posterior predictive distributions become equivalent. Our results have practical consequences: for instance, they show that optimal GP predictions in the sense of leave-one-out loss may occur at very large length-scales, which would be invisible to current implementations because of numerical difficulties.

翻译：高斯过程回归是贝叶斯统计中的基本工具，亦称克里金法，是频率学派核岭回归的贝叶斯对应方法。现有理论工作多聚焦于大样本渐近分析，即随着数据量增加时的渐近行为。除规则网格等简单情形外，固定样本分析困难重重。本研究基于Driscoll与Fornberg（2002）在逼近理论中首先提出的"平坦极限"概念，开展固定样本分析。在平坦极限渐近中，核心目标是刻画当核函数长度尺度趋于无穷大时，核方法在数据范围内呈现"平坦"特征的行为。令人惊讶的是，该极限不仅存在，且展现出有趣的特性：Driscoll与Fornberg发现，当核函数为高斯核时，径向基插值在平坦极限下收敛至多项式插值。后续研究表明，这一结论在多变量情形下依然成立，但非高斯核可能产生（多调和）样条函数作为极限插值。利用近期关于平坦极限下核矩阵谱行为的研究成果，我们考察高斯过程回归的平坦极限。结果表明，高斯过程回归在平坦极限下依据不同核函数趋向于（多变量）多项式回归或（多调和）样条回归。值得注意的是，这一结论同时适用于预测均值与预测方差，使得后验预测分布趋于等价。研究成果具有实际意义：例如，基于留一法损失的最优高斯过程预测可能在极大长度尺度处出现，而现有实现因数值困难无法观测到该现象。