Gaussian processes are flexible probabilistic regression models which are widely used in statistics and machine learning. However, a drawback is their limited scalability to large data sets. To alleviate this, full-scale approximations (FSAs) combine predictive process methods and covariance tapering, thus approximating both global and local structures. We show how iterative methods can be used to reduce computational costs in calculating likelihoods, gradients, and predictive distributions with FSAs. In particular, we introduce a novel preconditioner and show theoretically and empirically that it accelerates the conjugate gradient method's convergence speed and mitigates its sensitivity with respect to the FSA parameters and the eigenvalue structure of the original covariance matrix, and we demonstrate empirically that it outperforms a state-of-the-art pivoted Cholesky preconditioner. Furthermore, we introduce an accurate and fast way to calculate predictive variances using stochastic simulation and iterative methods. In addition, we show how our newly proposed fully independent training conditional (FITC) preconditioner can also be used in iterative methods for Vecchia approximations. In our experiments, it outperforms existing state-of-the-art preconditioners for Vecchia approximations. All methods are implemented in a free C++ software library with high-level Python and R packages.

翻译：高斯过程是一种灵活的概率回归模型，广泛应用于统计学和机器学习领域。然而，其可扩展性在处理大规模数据集时存在局限。为缓解这一问题，全尺度近似方法结合了预测过程方法与协方差锥化技术，从而同时近似全局与局部结构。本文展示了如何利用迭代方法降低FSA在计算似然函数、梯度及预测分布时的计算成本。特别地，我们提出了一种新型预处理器，从理论与实验两方面证明其能加速共轭梯度法的收敛速度，并降低其对FSA参数及原始协方差矩阵特征值结构的敏感性；实验表明该预处理器性能优于当前最先进的枢轴Cholesky预处理器。此外，我们提出了一种基于随机模拟与迭代方法的快速精确预测方差计算方案。进一步地，我们证明了新提出的完全独立训练条件预处理器同样适用于Vecchia近似的迭代计算，在实验中其性能优于现有Vecchia近似的最先进预处理器。所有方法均已在一个免费的C++软件库中实现，并提供高级Python与R语言接口。