Bayesian variable selection under spike-and-slab priors provides interpretable posterior inclusion probabilities for individual predictors, but exploring large discrete model spaces using conventional Markov chain Monte Carlo methods can be computationally challenging. We develop a collapsed importance-weighted variational framework for Gaussian linear regression. The framework analytically integrates out the remaining model parameters and directly approximates the posterior over binary inclusion indicators using a factorised Bernoulli distribution. To improve posterior inclusion probability estimation while retaining the computational simplicity of the factorised variational family, we replace the conventional evidence lower bound with a tighter variational Rényi importance-weighted lower bound. We establish a finite-sample upper bound on posterior inclusion probability approximation errors and show that, under suitable conditions, the optimised variational inclusion probabilities converge to their exact posterior counterparts as the number of importance samples increases and the Rényi parameter decreases appropriately. We further establish that analytical marginalisation tightens the variational objective and the corresponding inclusion probability error bound, while reducing importance-weight variance and improving the asymptotic efficiency of importance-sampling post-processing. To address the challenges of estimating gradients with respect to the variational parameters when the inclusion indicators are discrete, we leverage recent advances in variance-reduced score-function estimators, together with an annealing strategy to stabilise optimisation. Simulation experiments and empirical applications demonstrate improvements in posterior inclusion probability estimation relative to standard variational inference and two existing variational approaches.
翻译:暂无翻译