Schr\"odinger Bridge (SB) is an entropy-regularized optimal transport problem that has received increasing attention in deep generative modeling for its mathematical flexibility compared to the Scored-based Generative Model (SGM). However, it remains unclear whether the optimization principle of SB relates to the modern training of deep generative models, which often rely on constructing log-likelihood objectives.This raises questions on the suitability of SB models as a principled alternative for generative applications. In this work, we present a novel computational framework for likelihood training of SB models grounded on Forward-Backward Stochastic Differential Equations Theory - a mathematical methodology appeared in stochastic optimal control that transforms the optimality condition of SB into a set of SDEs. Crucially, these SDEs can be used to construct the likelihood objectives for SB that, surprisingly, generalizes the ones for SGM as special cases. This leads to a new optimization principle that inherits the same SB optimality yet without losing applications of modern generative training techniques, and we show that the resulting training algorithm achieves comparable results on generating realistic images on MNIST, CelebA, and CIFAR10. Our code is available at https://github.com/ghliu/SB-FBSDE.
翻译:薛定谔桥(Schrödinger Bridge, SB)是一种熵正则化的最优输运问题,因其在数学灵活性上优于基于得分的生成模型(SGM)而在深度生成建模领域受到日益关注。然而,SB的优化原理是否与依赖构建对数似然目标的现代深度生成模型训练方法存在关联,目前尚不明确。这引发了关于SB模型能否作为生成应用规范性替代方案的适用性质疑。本文提出了一种基于前向-后向随机微分方程理论(随机最优控制领域的一种数学方法,可将SB最优性条件转化为一组随机微分方程)的SB模型似然训练新型计算框架。关键在于,这些随机微分方程可用于构建SB的似然目标,且令人惊讶的是,这些目标能够将SGM的似然目标作为特例进行泛化。由此产生的新优化原理在继承SB最优性的同时,并未丧失现代生成训练技术的应用能力。实验表明,基于该原理的训练算法在MNIST、CelebA和CIFAR-10数据集上生成逼真图像方面取得了可比结果。我们的代码开源地址为https://github.com/ghliu/SB-FBSDE。