Gaussian processes are flexible probabilistic regression models which are widely used in statistics and machine learning. However, a drawback is their limited scalability to large data sets. To alleviate this, we consider full-scale approximations (FSAs) that combine predictive process methods and covariance tapering, thus approximating both global and local structures. We show how iterative methods can be used to reduce the computational costs for calculating likelihoods, gradients, and predictive distributions with FSAs. We introduce a novel preconditioner and show that it accelerates the conjugate gradient method's convergence speed and mitigates its sensitivity with respect to the FSA parameters and the eigenvalue structure of the original covariance matrix, and we demonstrate empirically that it outperforms a state-of-the-art pivoted Cholesky preconditioner. Further, we present a novel, accurate, and fast way to calculate predictive variances relying on stochastic estimations and iterative methods. In both simulated and real-world data experiments, we find that our proposed methodology achieves the same accuracy as Cholesky-based computations with a substantial reduction in computational time. Finally, we also compare different approaches for determining inducing points in predictive process and FSA models. All methods are implemented in a free C++ software library with high-level Python and R packages.

翻译：高斯过程是一种灵活的概率回归模型，在统计学和机器学习领域得到广泛应用。然而，其缺点在于处理大规模数据集时的可扩展性有限。为缓解此问题，本文研究结合预测过程方法与协方差锥化的全尺度逼近方法，从而同时逼近全局与局部结构。我们展示了如何利用迭代方法来降低使用全尺度逼近计算似然、梯度和预测分布的计算成本。我们提出了一种新颖的预处理器，并证明其能加速共轭梯度法的收敛速度，降低其对全尺度逼近参数及原始协方差矩阵特征值结构的敏感性；实证结果表明该预处理器优于目前最先进的枢轴Cholesky预处理器。此外，我们提出了一种基于随机估计与迭代方法的新颖、准确且快速的计算预测方差的方法。在模拟数据与真实数据实验中，我们发现所提方法在计算精度上与基于Cholesky分解的计算相当，同时计算时间显著减少。最后，我们还比较了在预测过程与全尺度逼近模型中确定诱导点的不同方法。所有方法均在一个免费的C++软件库中实现，并提供高级Python与R软件包。