We introduce a flexible empirical Bayes approach for fitting Bayesian generalized linear models. Specifically, we adopt a novel mean-field variational inference (VI) method and the prior is estimated within the VI algorithm, making the method tuning-free. Unlike traditional VI methods that optimize the posterior density function, our approach directly optimizes the posterior mean and prior parameters. This formulation reduces the number of parameters to optimize and enables the use of scalable algorithms such as L-BFGS and stochastic gradient descent. Furthermore, our method automatically determines the optimal posterior based on the prior and likelihood, distinguishing it from existing VI methods that often assume a Gaussian variational. Our approach represents a unified framework applicable to a wide range of exponential family distributions, removing the need to develop unique VI methods for each combination of likelihood and prior distributions. We apply the framework to solve sparse logistic regression and demonstrate the superior predictive performance of our method in extensive numerical studies, by comparing it to prevalent sparse logistic regression approaches.

翻译：本文提出了一种灵活的经验贝叶斯方法用于拟合贝叶斯广义线性模型。具体而言，我们采用了一种新颖的均值场变分推断方法，并在变分推断算法内部估计先验分布，从而使该方法无需调参。与优化后验密度函数的传统变分推断方法不同，我们的方法直接优化后验均值与先验参数。这种形式减少了待优化参数的数量，并使得可扩展算法（如L-BFGS和随机梯度下降）的应用成为可能。此外，我们的方法能根据先验和似然函数自动确定最优后验分布，这区别于通常假设高斯变分分布的现有变分推断方法。本方法构建了一个适用于广泛指数族分布的统一框架，无需为每种似然函数与先验分布的组合开发特定的变分推断方法。我们将该框架应用于稀疏逻辑回归问题，并通过与主流稀疏逻辑回归方法的对比，在大量数值实验中证明了本方法具有更优的预测性能。