Gaussian processes (GPs) offer appealing properties but are costly to train at scale. Sparse variational GP (SVGP) approximations reduce cost yet still rely on Cholesky decompositions of kernel matrices, ill-suited to low-precision, massively parallel hardware. While one can construct valid variational bounds that rely only on matrix multiplications (matmuls) via an auxiliary matrix parameter, optimising them with off-the-shelf first-order methods is challenging. We make the inverse-free approach practical by proposing a better-conditioned bound and deriving a matmul-only natural-gradient update for the auxiliary parameter, markedly improving stability and convergence. We further provide simple heuristics, such as step-size schedules and stopping criteria, that make the overall optimisation routine fit seamlessly into existing workflows. Across regression and classification benchmarks, we demonstrate that our method 1) serves as a drop-in replacement in SVGP-based models (e.g., deep GPs), 2) recovers similar performance to traditional methods, and 3) can be faster than baselines when well tuned.

翻译：高斯过程（GPs）具有吸引人的特性，但在大规模训练时成本高昂。稀疏变分高斯过程（SVGP）近似降低了计算成本，但仍依赖于核矩阵的Cholesky分解，这难以适配低精度、大规模并行硬件。尽管通过引入辅助矩阵参数可以构造仅依赖矩阵乘法（matmuls）的有效变分下界，但使用现成的一阶方法优化这些下界仍面临挑战。我们通过提出一个条件数更优的下界并推导出仅涉及矩阵乘法的自然梯度更新规则来优化辅助参数，从而显著提升了稳定性与收敛性，使免逆方法变得实用。我们进一步提供了简单的启发式策略（如步长调度和停止准则），使得整体优化流程能够无缝集成到现有工作流中。在回归与分类基准测试中，我们证明该方法：1）可作为SVGP基础模型（如深度GPs）的即插即用替代方案；2）能够恢复与传统方法相当的性能；3）在充分调参后可比基线方法更快。