The focus of this work is the convergence of non-stationary and deep Gaussian process regression. More precisely, we follow a Bayesian approach to regression or interpolation, where the prior placed on the unknown function $f$ is a non-stationary or deep Gaussian process, and we derive convergence rates of the posterior mean to the true function $f$ in terms of the number of observed training points. In some cases, we also show convergence of the posterior variance to zero. The only assumption imposed on the function $f$ is that it is an element of a certain reproducing kernel Hilbert space, which we in particular cases show to be norm-equivalent to a Sobolev space. Our analysis includes the case of estimated hyper-parameters in the covariance kernels employed, both in an empirical Bayes' setting and the particular hierarchical setting constructed through deep Gaussian processes. We consider the settings of noise-free or noisy observations on deterministic or random training points. We establish general assumptions sufficient for the convergence of deep Gaussian process regression, along with explicit examples demonstrating the fulfilment of these assumptions. Specifically, our examples require that the H\"older or Sobolev norms of the penultimate layer are bounded almost surely.

翻译：本文研究非平稳与深度高斯过程回归的收敛性问题。具体而言，我们遵循回归或插值的贝叶斯方法，其中未知函数$f$的先验分布设定为非平稳或深度高斯过程，并推导了在观测训练点数量下后验均值收敛至真实函数$f$的收敛速率。在某些情况下，我们还证明了后验方差收敛至零。对函数$f$施加的唯一假设是它属于某个再生核希尔伯特空间，我们在特定情形下证明了该空间与索伯列夫空间范数等价。我们的分析涵盖协方差核中超参数估计的情形，包括经验贝叶斯设置以及通过深度高斯过程构建的特定分层设置。我们考虑了无噪声或有噪声观测在确定性或随机训练点上的设置。建立了足以保证深度高斯过程回归收敛的通用假设条件，并给出明确示例验证这些假设的满足性。具体而言，我们的示例要求倒数第二层的赫尔德或索伯列夫范数以概率1有界。