Posterior sampling with the spike-and-slab prior [MB88], a popular multimodal distribution used to model uncertainty in variable selection, is considered the theoretical gold standard method for Bayesian sparse linear regression [CPS09, Roc18]. However, designing provable algorithms for performing this sampling task is notoriously challenging. Existing posterior samplers for Bayesian sparse variable selection tasks either require strong assumptions about the signal-to-noise ratio (SNR) [YWJ16], only work when the measurement count grows at least linearly in the dimension [MW24], or rely on heuristic approximations to the posterior. We give the first provable algorithms for spike-and-slab posterior sampling that apply for any SNR, and use a measurement count sublinear in the problem dimension. Concretely, assume we are given a measurement matrix $\mathbf{X} \in \mathbb{R}^{n\times d}$ and noisy observations $\mathbf{y} = \mathbf{X}\mathbf{\theta}^\star + \mathbf{\xi}$ of a signal $\mathbf{\theta}^\star$ drawn from a spike-and-slab prior $\pi$ with a Gaussian diffuse density and expected sparsity k, where $\mathbf{\xi} \sim \mathcal{N}(\mathbb{0}_n, \sigma^2\mathbf{I}_n)$. We give a polynomial-time high-accuracy sampler for the posterior $\pi(\cdot \mid \mathbf{X}, \mathbf{y})$, for any SNR $\sigma^{-1}$ > 0, as long as $n \geq k^3 \cdot \text{polylog}(d)$ and $X$ is drawn from a matrix ensemble satisfying the restricted isometry property. We further give a sampler that runs in near-linear time $\approx nd$ in the same setting, as long as $n \geq k^5 \cdot \text{polylog}(d)$. To demonstrate the flexibility of our framework, we extend our result to spike-and-slab posterior sampling with Laplace diffuse densities, achieving similar guarantees when $\sigma = O(\frac{1}{k})$ is bounded.

翻译：尖峰-平板先验[MB88]作为一种常用于变量选择不确定性建模的多峰分布，其对应的后验采样被公认为贝叶斯稀疏线性回归的理论黄金标准方法[CPS09, Roc18]。然而，为这一采样任务设计可证明的算法极具挑战性。现有针对贝叶斯稀疏变量选择任务的后验采样器要么需要对信噪比(SNR)施加强假设[YWJ16]，要么仅当测量数量至少随维度线性增长时有效[MW24]，或依赖于对后验的启发式近似。我们首次提出了适用于任意SNR、且所需测量数量低于问题维度线性的尖峰-平板后验采样可证明算法。具体而言，假设给定测量矩阵$\mathbf{X} \in \mathbb{R}^{n\times d}$和含噪观测$\mathbf{y} = \mathbf{X}\mathbf{\theta}^\star + \mathbf{\xi}$，其中信号$\mathbf{\theta}^\star$服从具有高斯扩散密度与期望稀疏度k的尖峰-平板先验$\pi$，噪声$\mathbf{\xi} \sim \mathcal{N}(\mathbb{0}_n, \sigma^2\mathbf{I}_n)$。我们提出一个多项式时间高精度采样器，可在任意SNR $\sigma^{-1}$ > 0条件下对后验$\pi(\cdot \mid \mathbf{X}, \mathbf{y})$进行采样，仅需满足$n \geq k^3 \cdot \text{polylog}(d)$且矩阵$X$来自满足限制等距性质的矩阵族。进一步，我们在相同设置下给出近线性时间$\approx nd$的采样器，仅需满足$n \geq k^5 \cdot \text{polylog}(d)$。为展示框架的灵活性，我们将结果扩展至拉普拉斯扩散密度的尖峰-平板后验采样，在$\sigma = O(\frac{1}{k})$有界时获得类似的理论保证。