Cyclical MCMC is a novel MCMC framework recently proposed by Zhang et al. (2019) to address the challenge posed by high-dimensional multimodal posterior distributions like those arising in deep learning. The algorithm works by generating a nonhomogeneous Markov chain that tracks -- cyclically in time -- tempered versions of the target distribution. We show in this work that cyclical MCMC converges to the desired probability distribution in settings where the Markov kernels used are fast mixing, and sufficiently long cycles are employed. However in the far more common settings of slow mixing kernels, the algorithm may fail to produce samples from the desired distribution. In particular, in a simple mixture example with unequal variance, we show by simulation that cyclical MCMC fails to converge to the desired limit. Finally, we show that cyclical MCMC typically estimates well the local shape of the target distribution around each mode, even when we do not have convergence to the target.

翻译：循环MCMC是Zhang等人（2019）近期提出的一种新型MCMC框架，旨在解决深度学习等场景中高维多模态后验分布带来的挑战。该算法通过生成一个非齐次马尔可夫链实现，该链在时间上循环跟踪目标分布的退火版本。本文证明：当使用的马尔可夫核具备快速混合特性且采用足够长的循环周期时，循环MCMC收敛于目标概率分布。然而在更常见的慢速混合核场景中，该算法可能无法从目标分布中生成样本。特别地，在具有不等方差的简单混合示例中，我们通过仿真证明循环MCMC未能收敛至目标极限。最后，我们证明即使未收敛到目标分布，循环MCMC通常也能较好地估计目标分布各模态周围的局部形态。