We establish a connection between stochastic optimal control and generative models based on stochastic differential equations (SDEs), such as recently developed diffusion probabilistic models. In particular, we derive a Hamilton-Jacobi-Bellman equation that governs the evolution of the log-densities of the underlying SDE marginals. This perspective allows to transfer methods from optimal control theory to generative modeling. First, we show that the evidence lower bound is a direct consequence of the well-known verification theorem from control theory. Further, we can formulate diffusion-based generative modeling as a minimization of the Kullback-Leibler divergence between suitable measures in path space. Finally, we develop a novel diffusion-based method for sampling from unnormalized densities -- a problem frequently occurring in statistics and computational sciences. We demonstrate that our time-reversed diffusion sampler (DIS) can outperform other diffusion-based sampling approaches on multiple numerical examples.

翻译：我们建立了随机最优控制与基于随机微分方程（SDE）的生成模型（如近期发展的扩散概率模型）之间的理论联系。具体而言，我们推导出一个控制底层SDE边际对数密度演化的Hamilton-Jacobi-Bellman方程。该视角使得最优控制理论中的方法能够迁移至生成建模领域。首先，我们证明证据下界是控制理论中著名验证定理的直接推论。其次，我们可将基于扩散的生成建模表述为路径空间中合适测度之间Kullback-Leibler散度的最小化问题。最后，我们提出一种从非归一化密度（统计与计算科学中常见问题）进行采样的新型扩散方法。通过多个数值算例，我们证明了所提出的时间反转扩散采样器（DIS）能够超越其他基于扩散的采样方法。