Despite the enormous success of Hamiltonian Monte Carlo and related Markov Chain Monte Carlo (MCMC) methods, sampling often still represents the computational bottleneck in scientific applications. Availability of parallel resources can significantly speed up MCMC inference by running a large number of chains in parallel, each collecting a single sample. However, the parallel approach converges slowly if the chains are not initialized close to the target distribution (cold start). Theoretically this can be resolved by initially running MCMC without Metropolis-Hastings adjustment to quickly converge to the vicinity of the target distribution and then turn on adjustment to achieve fine convergence. However, no practical scheme uses this strategy, due to the difficulty of automatically selecting the step size during the unadjusted phase. We here develop Late Adjusted Parallel Sampler (LAPS), which is precisely such a scheme and is applicable out of the box, all the hyperparameters are selected automatically. LAPS takes advantage of ensemble-based hyperparameter adaptation to estimate the bias at each iteration and converts it to the appropriate step size. We show that LAPS consistently and significantly outperforms ensemble adjusted methods such as MEADS or ChESS and the optimization-based initializer Pathfinder on a variety of standard benchmark problems. LAPS typically achieves two orders of magnitude lower wall-clock time than the corresponding sequential algorithms such as NUTS.

翻译：尽管哈密顿蒙特卡洛及其相关马尔可夫链蒙特卡洛方法取得了巨大成功，采样过程在科学应用中通常仍是计算瓶颈。利用并行资源运行大量并行链（每条链收集单个样本）可显著加速MCMC推断。然而，若链未初始化在目标分布附近（冷启动），这种并行方法收敛缓慢。理论上，可通过初始阶段运行未经Metropolis-Hastings调整的MCMC快速收敛至目标分布邻域，再启用调整以实现精细收敛来解决此问题。但由于未调整阶段步长自动选择的困难，现有方案均未采用此策略。本文提出延迟调整并行采样器，该方案可直接应用且所有超参数均自动选择。LAPS利用基于集成样本的超参数自适应技术估计每次迭代的偏差，并将其转换为合适步长。实验表明，在多种标准基准问题上，LAPS持续显著优于集成调整方法（如MEADS或ChESS）和基于优化的初始化器Pathfinder。相较于NUTS等顺序算法，LAPS通常可获得两个数量级更低的实际运行时间。