We study the implementation of Automatic Differentiation Variational inference (ADVI) for Bayesian inference on regression models with bridge penalization. The bridge approach uses $\ell_{\alpha}$ norm, with $\alpha \in (0, +\infty)$ to define a penalization on large values of the regression coefficients, which includes the Lasso ($\alpha = 1$) and ridge $(\alpha = 2)$ penalizations as special cases. Full Bayesian inference seamlessly provides joint uncertainty estimates for all model parameters. Although MCMC aproaches are available for bridge regression, it can be slow for large dataset, specially in high dimensions. The ADVI implementation allows the use of small batches of data at each iteration (due to stochastic gradient based algorithms), therefore speeding up computational time in comparison with MCMC. We illustrate the approach on non-parametric regression models with B-splines, although the method works seamlessly for other choices of basis functions. A simulation study shows the main properties of the proposed method.

翻译：本研究探讨了自动微分变分推断（ADVI）在带桥惩罚的回归模型贝叶斯推断中的实现。桥方法通过 $\ell_{\alpha}$ 范数（其中 $\alpha \in (0, +\infty)$）对回归系数的大值施加惩罚，其特例包括Lasso（$\alpha = 1$）和岭回归（$\alpha = 2$）惩罚。完全贝叶斯推断能够无缝地为所有模型参数提供联合不确定性估计。尽管桥回归已有适用的MCMC方法，但在大数据集（尤其是高维情形）下计算速度较慢。ADVI的实现允许每次迭代使用小批量数据（基于随机梯度算法），从而相比MCMC加快计算时间。我们以B样条非参数回归模型为例展示该方法，但其同样适用于其他基函数选择。模拟研究揭示了所提方法的主要性质。