Deep generative models based on neural differential equations have quickly become the state-of-the-art for numerous generation tasks across many different applications. These models rely on ODE/SDE solvers which integrate from a prior distribution to the data distribution. In many applications it is highly desirable to then integrate in the other direction. The standard solvers, however, accumulate discretization errors which don't align with the forward trajectory, thereby prohibiting an exact inversion. In applications where the precision of the generative model is paramount this inaccuracy in inversion is often unacceptable. Current approaches to solving the inversion of these models results in significant downstream issues with poor stability and low-order of convergence; moreover, they are strictly limited to the ODE domain. In this work, we propose a new family of reversible exponential (stochastic) Runge-Kutta solvers which we refer to as Rex developed by an application of Lawson methods to convert any explicit (stochastic) Runge-Kutta scheme into a reversible one. In addition to a rigorous theoretical analysis of the proposed solvers, we also empirically demonstrate the utility of Rex on improving the sampling of Boltzmann distributions with flow models, and improving image generation and editing capabilities with diffusion models.

翻译：基于神经微分方程的深度生成模型已迅速成为众多应用领域中多种生成任务的最先进方法。这些模型依赖于从先验分布积分至数据分布的常微分方程/随机微分方程求解器。在许多应用中，随后进行反向积分是极为必要的。然而，标准求解器会累积与正向轨迹不一致的离散化误差，从而无法实现精确的逆向求解。在生成模型精度至关重要的应用中，这种逆向求解的不准确性通常是不可接受的。当前针对这些模型逆向求解的方法存在显著的稳定性差与收敛阶数低等下游问题；此外，这些方法严格局限于常微分方程领域。本文提出了一类新的可逆指数（随机）龙格-库塔求解器，我们称之为Rex，其通过应用Lawson方法将任意显式（随机）龙格-库塔格式转化为可逆格式而构建。除了对所提求解器进行严格的理论分析外，我们还通过实验证明了Rex在提升基于流模型的玻尔兹曼分布采样能力，以及增强基于扩散模型的图像生成与编辑性能方面的实用性。