Gaussian processes are flexible probabilistic regression models which are widely used in statistics and machine learning. However, a drawback is their limited scalability to large data sets. To alleviate this, we consider full-scale approximations (FSAs) that combine predictive process methods and covariance tapering, thus approximating both global and local structures. We show how iterative methods can be used to reduce the computational costs for calculating likelihoods, gradients, and predictive distributions with FSAs. We introduce a novel preconditioner and show that it accelerates the conjugate gradient method's convergence speed and mitigates its sensitivity with respect to the FSA parameters and the eigenvalue structure of the original covariance matrix, and we demonstrate empirically that it outperforms a state-of-the-art pivoted Cholesky preconditioner. Further, we present a novel, accurate, and fast way to calculate predictive variances relying on stochastic estimations and iterative methods. In both simulated and real-world data experiments, we find that our proposed methodology achieves the same accuracy as Cholesky-based computations with a substantial reduction in computational time. Finally, we also compare different approaches for determining inducing points in predictive process and FSA models. All methods are implemented in a free C++ software library with high-level Python and R packages.

翻译：高斯过程是一种灵活的统计回归模型，在统计学和机器学习领域应用广泛。然而，该方法在处理大规模数据集时存在可扩展性不足的局限。为缓解此问题，本文研究结合预测过程方法与协方差锥化的全尺度近似方法，从而同时逼近全局与局部结构。我们展示了如何利用迭代方法降低全尺度近似下似然函数、梯度及预测分布的计算成本。本文提出了一种新型预处理器，证明其能加速共轭梯度法的收敛速度，并降低算法对全尺度近似参数及原始协方差矩阵特征值结构的敏感性；实证研究表明该预处理器性能优于当前最先进的枢轴Cholesky预处理器。此外，我们提出了一种基于随机估计与迭代方法的创新方案，能够快速精确地计算预测方差。在模拟数据与真实数据实验中，本研究所提方法在保持与基于Cholesky分解计算相同精度的同时，显著降低了计算时间。最后，我们比较了预测过程与全尺度近似模型中确定诱导点的不同方法。所有算法均已在开源C++软件库中实现，并提供高级Python与R语言接口。