Bayesian methods provide an elegant framework for estimating parameter posteriors and quantification of uncertainty associated with probabilistic models. However, they often suffer from slow inference times. To address this challenge, Bayesian Pseudo-Coresets (BPC) have emerged as a promising solution. BPC methods aim to create a small synthetic dataset, known as pseudo-coresets, that approximates the posterior inference achieved with the original dataset. This approximation is achieved by optimizing a divergence measure between the true posterior and the pseudo-coreset posterior. Various divergence measures have been proposed for constructing pseudo-coresets, with forward Kullback-Leibler (KL) divergence being the most successful. However, using forward KL divergence necessitates sampling from the pseudo-coreset posterior, often accomplished through approximate Gaussian variational distributions. Alternatively, one could employ Markov Chain Monte Carlo (MCMC) methods for sampling, but this becomes challenging in high-dimensional parameter spaces due to slow mixing. In this study, we introduce a novel approach for constructing pseudo-coresets by utilizing contrastive divergence. Importantly, optimizing contrastive divergence eliminates the need for approximations in the pseudo-coreset construction process. Furthermore, it enables the use of finite-step MCMC methods, alleviating the requirement for extensive mixing to reach a stationary distribution. To validate our method's effectiveness, we conduct extensive experiments on multiple datasets, demonstrating its superiority over existing BPC techniques.

翻译：摘要：贝叶斯方法为估计参数后验分布及量化概率模型相关的不确定性提供了优雅的框架。然而，这类方法常面临推断速度缓慢的挑战。为解决此问题，贝叶斯伪核集（BPC）作为一种有前景的解决方案应运而生。BPC方法旨在创建一种小型合成数据集（即伪核集），使其能够近似原始数据集所实现的后验推断。这种近似通过优化真实后验与伪核集后验之间的散度度量来实现。目前已提出多种用于构建伪核集的散度度量，其中前向KL散度最为成功。但使用前向KL散度需从伪核集后验中采样，通常需借助近似高斯变分分布完成。另一种方案是采用马尔可夫链蒙特卡洛方法进行采样，但高维参数空间中链的缓慢混合会使采样变得困难。本研究提出一种利用对比散度构建伪核集的新方法。关键在于，优化对比散度无需在伪核集构建过程中引入近似假设。此外，该方法支持使用有限步马尔可夫链蒙特卡洛方法，从而降低了对链充分混合以达平稳分布的要求。为验证方法的有效性，我们在多个数据集上开展了广泛实验，结果表明该方法显著优于现有BPC技术。