Gaussian processes (GPs) provide a principled Bayesian framework for uncertainty estimation, but their computational complexity severely limits scalability to large datasets. We propose SIKA-GP, which accelerates GP inference using sparse inducing kernel approximations based on a dyadic ordered template basis, incurring only ${O}(\log M)$ complexity dependence on the number of inducing points. Our approach constructs compact and expressive kernel representations from sparsely activated bases, enabling efficient tensorized GPU computation and seamless integration with modern large-scale models. SIKA-GP can be naturally embedded into Bayesian neural networks (BNNs) with sparse activations, yielding significant speedups in both training and inference without sacrificing predictive performance. The method naturally extends to deep feature learning, addressing the scalability challenges introduced by deep architectures and high-dimensional feature representations. Empirical results on vision and transformer-based language benchmarks demonstrate that our approach consistently delivers fast and accurate GP models, providing a principled path toward scalable kernel learning.

翻译：高斯过程（GPs）为不确定性估计提供了规范的贝叶斯框架，但其计算复杂性严重限制了在大规模数据集上的可扩展性。我们提出SIKA-GP方法，采用基于二元有序模板基的稀疏诱导核近似来加速高斯过程推理，其计算复杂度仅为${O}(\log M)$（其中M为诱导点数量）。该方法通过稀疏激活基构建紧凑且表达能力强的核表示，实现了高效的张量化GPU计算，并能无缝集成现代大规模模型。SIKA-GP可自然嵌入具有稀疏激活机制的贝叶斯神经网络（BNNs），在不牺牲预测性能的前提下显著加速训练与推理过程。该方法自然地扩展至深度特征学习，有效解决了深度架构和高维特征表示带来的可扩展性挑战。在视觉任务和基于Transformer的语言基准测试上的实验结果表明，我们的方法始终能提供快速且精准的GP模型，为可扩展核学习开辟了规范化的实现路径。