Langevin Dynamics is a Stochastic Differential Equation (SDE) central to sampling and generative modeling and is implemented via time discretization. Langevin Monte Carlo (LMC), based on the Euler-Maruyama discretization, is the simplest and most studied algorithm. LMC can suffer from slow convergence - requiring a large number of steps of small step-size to obtain good quality samples. This becomes stark in the case of diffusion models where a large number of steps gives the best samples, but the quality degrades rapidly with smaller number of steps. Randomized Midpoint Method has been recently proposed as a better discretization of Langevin dynamics for sampling from strongly log-concave distributions. However, important applications such as diffusion models involve non-log concave densities and contain time varying drift. We propose its variant, the Poisson Midpoint Method, which approximates a small step-size LMC with large step-sizes. We prove that this can obtain a quadratic speed up of LMC under very weak assumptions. We apply our method to diffusion models for image generation and show that it maintains the quality of DDPM with 1000 neural network calls with just 50-80 neural network calls and outperforms ODE based methods with similar compute.

翻译：朗格文动力学作为一种随机微分方程，在采样与生成建模中居于核心地位，通常通过时间离散化实现。基于欧拉-丸山离散化的朗格文蒙特卡洛是最基础且研究最广泛的算法。LMC存在收敛速度缓慢的缺陷——需要大量小步长迭代才能获得高质量样本。这在扩散模型中尤为突出：大量步数能产生最优样本，但步数减少时样本质量会急剧下降。随机中点方法近期被提出，作为从强对数凹分布采样时朗格文动力学的更优离散化方案。然而在扩散模型等重要应用中，密度函数具有非对数凹性且包含时变漂移项。我们提出其变体——泊松中点方法，该方法能以大步长逼近小步长LMC。我们证明该方法在极弱假设下可实现LMC的二次加速。我们将本方法应用于图像生成的扩散模型，实验表明仅需50-80次神经网络调用即可保持DDPM使用1000次调用时的生成质量，且在相同计算量下优于基于ODE的方法。