We introduce Sparse Variational Bayesian Monte Carlo (SVBMC), a method for fast "post-process" Bayesian inference for models with black-box and potentially noisy likelihoods. SVBMC reuses all existing target density evaluations -- for example, from previous optimizations or partial Markov Chain Monte Carlo runs -- to build a sparse Gaussian process (GP) surrogate model of the log posterior density. Uncertain regions of the surrogate are then refined via active learning as needed. Our work builds on the Variational Bayesian Monte Carlo (VBMC) framework for sample-efficient inference, with several novel contributions. First, we make VBMC scalable to a large number of pre-existing evaluations via sparse GP regression, deriving novel Bayesian quadrature formulae and acquisition functions for active learning with sparse GPs. Second, we introduce noise shaping, a general technique to induce the sparse GP approximation to focus on high posterior density regions. Third, we prove theoretical results in support of the SVBMC refinement procedure. We validate our method on a variety of challenging synthetic scenarios and real-world applications. We find that SVBMC consistently builds good posterior approximations by post-processing of existing model evaluations from different sources, often requiring only a small number of additional density evaluations.

翻译：我们提出了稀疏变分贝叶斯蒙特卡洛（SVBMC），一种用于对具有黑箱且可能含噪似然函数的模型进行快速“后处理”贝叶斯推断的方法。SVBMC复用所有已有的目标密度评估结果——例如来自先前优化或部分马尔可夫链蒙特卡洛运行的结果——来构建对数后验密度的稀疏高斯过程（GP）替代模型。随后，通过主动学习按需精炼替代模型的不确定区域。我们的工作基于变分贝叶斯蒙特卡洛（VBMC）框架以实现样本高效推断，并包含若干创新贡献。首先，通过稀疏GP回归，我们推导了新的贝叶斯求积公式和用于稀疏GP主动学习的采集函数，从而使VBMC可扩展至大量预存评估数据。其次，我们引入了噪声整形技术，这是一种引导稀疏GP近似聚焦于高后验密度区域的通用方法。第三，我们证明了支持SVBMC精炼过程的理论结果。我们在多种具有挑战性的合成场景和实际应用中对方法进行了验证。结果表明，SVBMC通过后处理来自不同来源的已有模型评估结果，能够持续构建良好的后验近似，且通常仅需少量额外的密度评估。