Gaussian processes (GP) regression has gained substantial popularity in machine learning applications. The behavior of a GP regression depends on the choice of covariance function. Stationary covariance functions are favorite in machine learning applications. However, (non-periodic) stationary covariance functions are always mean reverting and can therefore exhibit pathological behavior when applied to data that does not relax to a fixed global mean value. In this paper, we show that it is possible to use improper GP prior with infinite variance to define processes that are stationary but not mean reverting. To this aim, we introduce a large class of improper kernels that can only be defined in this improper regime. Specifically, we introduce the Smooth Walk kernel, which produces infinitely smooth samples, and a family of improper Mat\'ern kernels, which can be defined to be $j$-times differentiable for any integer $j$. The resulting posterior distributions can be computed analytically and it involves a simple correction of the usual formulas. By analyzing both synthetic and real data, we demonstrate that these improper kernels solve some known pathologies of mean reverting GP regression while retaining most of the favourable properties of ordinary smooth stationary kernels.

翻译：高斯过程（GP）回归在机器学习应用中已获得广泛关注。GP回归的行为取决于协方差函数的选择。平稳协方差函数是机器学习应用中的首选。然而，（非周期）平稳协方差函数始终具有均值回归特性，因此在应用于未松弛到固定全局均值的数据时可能表现出病态行为。本文证明，利用具有无限方差的非正常GP先验，可以定义平稳但无均值回归特性的过程。为此，我们引入了一大类仅能在非正常状态下定义的非正常核函数。具体而言，我们提出了产生无限光滑样本的光滑漫步核函数，以及可定义为对任意整数j具有j次可微性的非正常Matérn核函数族。所得后验分布可通过解析计算，涉及对标准公式的简单修正。通过分析合成数据与真实数据，我们证明这些非正常核函数在保留普通光滑平稳核函数大部分优良性质的同时，解决了均值回归GP回归的某些已知病态问题。