Sparse Bayesian Learning (SBL) constructs an extremely sparse probabilistic model with very competitive generalization. However, SBL needs to invert a big covariance matrix with complexity $O(M^3)$ (M: feature size) for updating the regularization priors, making it difficult for problems with high dimensional feature space or large data size. As it may easily suffer from the memory overflow issue in such problems. This paper addresses this issue with a newly proposed diagonal Quasi-Newton (DQN) method for SBL called DQN-SBL where the inversion of big covariance matrix is ignored so that the complexity is reduced to $O(M)$. The DQN-SBL is thoroughly evaluated for non linear and linear classifications with various benchmarks of different sizes. Experimental results verify that DQN-SBL receives competitive generalization with a very sparse model and scales well to large-scale problems.

翻译：稀疏贝叶斯学习（SBL）构建了具有极强泛化能力的极稀疏概率模型。然而，SBL在更新正则化先验时需要求逆一个复杂度为$O(M^3)$（M：特征维度）的大规模协方差矩阵，这使其难以应对高维特征空间或大数据量问题，且在此类问题中极易遭遇内存溢出。本文针对该问题提出一种基于对角拟牛顿法（DQN）的新型SBL算法——DQN-SBL，通过避免大规模协方差矩阵求逆，将复杂度降至$O(M)$。针对非线性与线性分类任务，我们使用多种不同规模的基准数据集对DQN-SBL进行全面评估。实验结果验证，DQN-SBL在保持极稀疏模型的同时具有竞争性的泛化性能，并可良好扩展至大规模问题。