We develop a modular approach to Markov chain Monte Carlo (MCMC) sampling for unnormalized target densities. In this approach, Markov chains are constructed in parallel, each constrained to a subset of the target space. The Monte Carlo estimates from the constrained chains are then combined with appropriate weights, calculated from the transition probabilities between subsets. In addition to the computational advantages arising from its parallelized structure, this modular MCMC approach enables variance reduction for Monte Carlo estimation in settings where sampling from low-density regions is required. We develop a central limit theorem-type result for the resulting Monte Carlo estimates and propose a method for estimating their standard errors. Furthermore, by applying this modular sampling technique to simulated tempering, we propose a method for Monte Carlo estimation of expectations with respect to multimodal target distributions. This approach effectively addresses a well-known challenge of tempering-based methods: sampling efficiency can be greatly reduced when separated modes of the target distribution have different scales. We demonstrate the efficiency of the proposed methods through numerical examples, including one arising from Bayesian sparse regression with a spike-and-slab prior.

翻译：摘要：针对非归一化目标密度，我们提出了一种模块化马尔可夫链蒙特卡洛（MCMC）采样方法。在该方法中，并行构建多个马尔可夫链，每条链被约束在目标空间的某个子集内。随后，利用从子集间转移概率计算出的适当权重，将约束链的蒙特卡洛估计结果进行组合。除了并行化结构带来的计算优势外，该模块化MCMC方法还能在需要从低密度区域采样时实现蒙特卡洛估计的方差缩减。我们针对所得的蒙特卡洛估计量发展了中心极限定理型结果，并提出了一种估计其标准误的方法。此外，通过将该模块化采样技术应用于模拟退火，我们提出了一种针对多模态目标分布期望的蒙特卡洛估计方法。该方法有效解决了退火类方法中一个众所周知的挑战：当目标分布的分离模态具有不同尺度时，采样效率会大幅降低。我们通过数值示例（包括一个基于尖峰-平板先验的贝叶斯稀疏回归示例）展示了所提出方法的有效性。