Hamiltonian Monte Carlo (HMC) is the mainstay of applied Bayesian inference for differentiable models. However, HMC still struggles to sample from hierarchical models that induce densities with multiscale geometry: a large step size is needed to efficiently explore low curvature regions while a small step size is needed to accurately explore high curvature regions. We introduce the delayed rejection generalized HMC (DR-G-HMC) sampler that overcomes this challenge by employing dynamic step size selection, inspired by differential equation solvers. In generalized HMC, each iteration does a single leapfrog step. DR-G-HMC sequentially makes proposals with geometrically decreasing step sizes upon rejection of earlier proposals. This simulates Hamiltonian dynamics that can adjust its step size along a (stochastic) Hamiltonian trajectory to deal with regions of high curvature. DR-G-HMC makes generalized HMC competitive by decreasing the number of rejections which otherwise cause inefficient backtracking and prevents directed movement. We present experiments to demonstrate that DR-G-HMC (1) correctly samples from multiscale densities, (2) makes generalized HMC methods competitive with the state of the art No-U-Turn sampler, and (3) is robust to tuning parameters.

翻译：哈密顿蒙特卡洛（HMC）是可微模型应用贝叶斯推断的主要方法。然而，对于具有多尺度几何特征的层次模型所诱导的密度分布，HMC 仍难以有效采样：探索低曲率区域需要较大的步长，而准确探索高曲率区域则需要较小的步长。本文提出的延迟拒绝广义 HMC（DR-G-HMC）采样器通过采用受微分方程求解器启发的动态步长选择机制克服了这一挑战。在广义 HMC 中，每次迭代执行单次蛙跳步。DR-G-HMC 在先前提案被拒绝后，会按几何递减的步长序列生成新提案。这模拟了能够沿（随机）哈密顿轨迹调整步长以处理高曲率区域的哈密顿动力学。通过减少导致低效回溯和阻碍定向运动的拒绝次数，DR-G-HMC 使广义 HMC 具备了竞争力。实验表明 DR-G-HMC 具有以下特性：（1）能正确地从多尺度密度中采样；（2）使广义 HMC 方法与当前最先进的 No-U-Turn 采样器性能相当；（3）对调参具有鲁棒性。