We establish a connection between stochastic optimal control and generative models based on stochastic differential equations (SDEs), such as recently developed diffusion probabilistic models. In particular, we derive a Hamilton-Jacobi-Bellman equation that governs the evolution of the log-densities of the underlying SDE marginals. This perspective allows to transfer methods from optimal control theory to generative modeling. First, we show that the evidence lower bound is a direct consequence of the well-known verification theorem from control theory. Further, we can formulate diffusion-based generative modeling as a minimization of the Kullback-Leibler divergence between suitable measures in path space. Finally, we develop a novel diffusion-based method for sampling from unnormalized densities -- a problem frequently occurring in statistics and computational sciences. We demonstrate that our time-reversed diffusion sampler (DIS) can outperform other diffusion-based sampling approaches on multiple numerical examples.

翻译：我们建立了随机最优控制与基于随机微分方程（SDE）的生成模型（如最近发展的扩散概率模型）之间的联系。具体而言，我们推导出一个Hamilton-Jacobi-Bellman方程，该方程控制着底层SDE边际的对数密度的演化。这一视角使得我们可以将最优控制理论中的方法迁移到生成建模中。首先，我们证明证据下界是控制理论中众所周知的验证定理的直接推论。进一步地，我们可将基于扩散的生成建模表述为路径空间中合适测度之间的Kullback-Leibler散度最小化问题。最后，我们开发了一种基于扩散的新型方法，用于从未归一化密度中采样——这是统计学和计算科学中经常遇到的问题。我们通过多个数值实例证明，所提出的时间反扩散采样器（DIS）能够超越其他基于扩散的采样方法。