Gaussian processes are flexible, probabilistic, non-parametric models widely used in machine learning and statistics. However, their scalability to large data sets is limited by computational constraints. To overcome these challenges, we propose Vecchia-inducing-points full-scale (VIF) approximations combining the strengths of global inducing points and local Vecchia approximations. Vecchia approximations excel in settings with low-dimensional inputs and moderately smooth covariance functions, while inducing point methods are better suited to high-dimensional inputs and smoother covariance functions. Our VIF approach bridges these two regimes by using an efficient correlation-based neighbor-finding strategy for the Vecchia approximation of the residual process, implemented via a modified cover tree algorithm. We further extend our framework to non-Gaussian likelihoods by introducing iterative methods that substantially reduce computational costs for training and prediction by several orders of magnitudes compared to Cholesky-based computations when using a Laplace approximation. In particular, we propose and compare novel preconditioners and provide theoretical convergence results. Extensive numerical experiments on simulated and real-world data sets show that VIF approximations are both computationally efficient as well as more accurate and numerically stable than state-of-the-art alternatives. All methods are implemented in the open source C++ library GPBoost with high-level Python and R interfaces.

翻译：高斯过程是机器学习与统计学中广泛使用的灵活概率非参数模型，但其大规模数据可扩展性受计算约束限制。为克服这些挑战，我们提出融合全局诱导点与局部Vecchia近似优势的Vecchia诱导点全尺度（VIF）近似方法。Vecchia近似在低维输入与中等光滑协方差函数场景中表现优异，而诱导点方法更适用于高维输入与更光滑协方差函数。我们的VIF方法通过为残差过程的Vecchia近似设计高效的相关性邻域搜索策略（基于改进的覆盖树算法实现），弥合了这两种范式。进一步地，我们将框架扩展至非高斯似然场景，引入迭代方法——结合拉普拉斯近似时，可较基于Cholesky分解的计算将训练与预测计算成本降低数个数量级。特别地，我们提出并比较了新型预条件子的性能，并提供理论收敛性证明。在模拟与真实数据集上的大量数值实验表明，VIF近似在保持计算高效性的同时，较现有最优方法具有更高精度与数值稳定性。所有方法均已在开源C++库GPBoost中实现，并提供高级Python与R接口。