Gaussian processes are flexible, probabilistic, non-parametric models widely used in machine learning and statistics. However, their scalability to large data sets is limited by computational constraints. To overcome these challenges, we propose Vecchia-inducing-points full-scale (VIF) approximations combining the strengths of global inducing points and local Vecchia approximations. Vecchia approximations excel in settings with low-dimensional inputs and moderately smooth covariance functions, while inducing point methods are better suited to high-dimensional inputs and smoother covariance functions. Our VIF approach bridges these two regimes by using an efficient correlation-based neighbor-finding strategy for the Vecchia approximation of the residual process, implemented via a modified cover tree algorithm. We further extend our framework to non-Gaussian likelihoods by introducing iterative methods that substantially reduce computational costs for training and prediction by several orders of magnitudes compared to Cholesky-based computations when using a Laplace approximation. In particular, we propose and compare novel preconditioners and provide theoretical convergence results. Extensive numerical experiments on simulated and real-world data sets show that VIF approximations are both computationally efficient as well as more accurate and numerically stable than state-of-the-art alternatives. All methods are implemented in the open source C++ library GPBoost with high-level Python and R interfaces.

翻译：高斯过程是机器学习与统计学中广泛使用的灵活、概率性、非参数模型。然而，其在大规模数据集上的可扩展性受限于计算约束。为克服这些挑战，我们提出了Vecchia诱导点全尺度（VIF）近似方法，该方法结合了全局诱导点与局部Vecchia近似的优势。Vecchia近似在低维输入与中等平滑协方差函数的场景中表现优异，而诱导点方法更适用于高维输入及更平滑的协方差函数。我们的VIF方法通过采用基于相关性的高效邻域搜索策略（通过改进的覆盖树算法实现）对残差过程进行Vecchia近似，从而桥接这两种机制。我们进一步将该框架扩展至非高斯似然场景，引入了迭代方法，在使用拉普拉斯近似时，相比基于Cholesky分解的计算，这些方法将训练与预测的计算成本降低了数个数量级。特别地，我们提出并比较了新颖的预条件子，并提供了理论收敛性结果。在模拟与真实数据集上的大量数值实验表明，VIF近似不仅计算高效，而且比现有先进方法更精确、数值更稳定。所有方法均在开源C++库GPBoost中实现，并提供高级Python与R接口。