Finding effective ways to exploit parallel computing to accelerate Markov chain Monte Carlo methods is an important problem in Bayesian computation and related disciplines. In this paper, we consider the zeroth-order setting where the unnormalized target distribution can be evaluated but its gradient is unavailable for theoretical, practical, or computational reasons. We also assume access to $m$ parallel processors to accelerate convergence. The proposed approach is inspired by modern zeroth-order optimization methods, which mimic gradient-based schemes by replacing the gradient with a zeroth-order stochastic gradient estimator. Our contribution is twofold. First, we show that a naive application of popular zeroth-order stochastic gradient estimators within Markov chain Monte Carlo methods leads to algorithms with poor dependence on $m$, both for unadjusted and Metropolis-adjusted schemes. We then propose a simple remedy to this problem, based on a random-slice perspective, as opposed to a stochastic gradient one, obtaining a new class of zeroth-order samplers that provably achieve a polynomial speed-up in $m$. Theoretical findings are supported by numerical studies.

翻译：在贝叶斯计算及相关领域中，如何有效利用并行计算加速马尔可夫链蒙特卡洛方法是一个重要问题。本文研究零阶设定，即目标分布虽可计算但其梯度因理论、实际或计算原因不可用。同时假设可使用$m$个并行处理器以加速收敛。所提方法受现代零阶优化方法启发，通过用零阶随机梯度估计器替代梯度来模拟基于梯度的方案。我们的贡献有两点：首先，我们证明将流行的零阶随机梯度估计器直接应用于马尔可夫链蒙特卡洛方法时，无论对于未调整方案还是Metropolis调整方案，都会产生对$m$依赖性的算法性能缺陷。随后基于随机切片视角（而非随机梯度视角）提出简易改进方案，获得了一类新的零阶采样器，理论上可证明实现$m$的多项式级加速。数值研究验证了理论结论。