Diffusion generative modelling (DGM) based on stochastic differential equations (SDEs) with score matching has achieved unprecedented results in data generation. In this paper, we propose a novel fast high-quality generative modelling method based on high-order Langevin dynamics (HOLD) with score matching. This motive is proved by third-order Langevin dynamics. By augmenting the previous SDEs, e.g. variance exploding or variance preserving SDEs for single-data variable processes, HOLD can simultaneously model position, velocity, and acceleration, thereby improving the quality and speed of the data generation at the same time. HOLD is composed of one Ornstein-Uhlenbeck process and two Hamiltonians, which reduce the mixing time by two orders of magnitude. Empirical experiments for unconditional image generation on the public data set CIFAR-10 and CelebA-HQ show that the effect is significant in both Frechet inception distance (FID) and negative log-likelihood, and achieves the state-of-the-art FID of 1.85 on CIFAR-10.

翻译：基于随机微分方程（SDE）与分数匹配的扩散生成式建模（DGM）在数据生成中取得了前所未有的成果。本文提出一种基于高阶朗之万动力学（HOLD）与分数匹配的新型快速高质量生成式建模方法。该动机通过三阶朗之万动力学得以证明。通过增强先前的SDE（例如针对单数据变量过程的方差爆炸或方差保持SDE），HOLD可同时对位置、速度和加速度进行建模，从而同时提升数据生成的质量与速度。HOLD由一个奥恩斯坦-乌伦贝克过程和两个哈密顿量组成，将混合时间降低两个数量级。在公开数据集CIFAR-10和CelebA-HQ上的无条件图像生成实验表明，该方法在弗雷歇初始距离（FID）和负对数似然上均效果显著，并在CIFAR-10上实现了1.85的当前最优FID值。