Learning expressive kernels while retaining tractable inference remains a central challenge in scaling Gaussian processes (GPs) to large and complex datasets. We propose a scalable GP regressor based on deep basis kernels (DBKs). Our DBK is constructed from a small set of neural-network-parameterized basis functions with an explicit low-rank structure. This formulation immediately enables linear-complexity inference with respect to the number of samples, possibly without inducing points. DBKs provide a unifying perspective that recovers sparse deep kernel learning and Gaussian Bayesian last-layer methods as special cases. We further identify that naively maximizing the marginal likelihood can lead to oversimplified uncertainty and rank-deficient solutions. To address this, we introduce a mini-batch stochastic objective that directly targets the predictive distribution with decoupled regularization. Empirically, DBKs show advantages in predictive accuracy, uncertainty quantification, and computational efficiency across a range of large-scale regression benchmarks.

翻译：在将高斯过程（GPs）扩展到大规模复杂数据集时，学习表达能力强的核函数同时保持推断的可处理性，仍然是一个核心挑战。我们提出了一种基于深度基核（DBKs）的可扩展高斯过程回归器。我们的深度基核由一小组具有显式低秩结构的神经网络参数化基函数构建而成。这种构建方式直接实现了相对于样本数量的线性复杂度推断，且可能无需使用诱导点。深度基核提供了一个统一的视角，将稀疏深度核学习与高斯贝叶斯最后一层方法作为其特例进行恢复。我们进一步发现，简单地最大化边缘似然可能导致过度简化的不确定性以及秩亏缺的解。为解决此问题，我们引入了一种小批量随机优化目标，该目标直接针对预测分布，并采用解耦的正则化方法。实验表明，深度基核在一系列大规模回归基准测试中，在预测准确性、不确定性量化和计算效率方面均展现出优势。