Implicit generative modeling (IGM) aims to produce samples of synthetic data matching the characteristics of a target data distribution. Recent work (e.g. score-matching networks, diffusion models) has approached the IGM problem from the perspective of pushing synthetic source data toward the target distribution via dynamical perturbations or flows in the ambient space. In this direction, we present the score difference (SD) between arbitrary target and source distributions as a flow that optimally reduces the Kullback-Leibler divergence between them while also solving the Schroedinger bridge problem. We apply the SD flow to convenient proxy distributions, which are aligned if and only if the original distributions are aligned. We demonstrate the formal equivalence of this formulation to denoising diffusion models under certain conditions. We also show that the training of generative adversarial networks includes a hidden data-optimization sub-problem, which induces the SD flow under certain choices of loss function when the discriminator is optimal. As a result, the SD flow provides a theoretical link between model classes that individually address the three challenges of the "generative modeling trilemma" -- high sample quality, mode coverage, and fast sampling -- thereby setting the stage for a unified approach.

翻译：隐式生成建模（IGM）旨在生成与目标数据分布特征相匹配的合成数据样本。近期研究（例如分数匹配网络、扩散模型）从通过环境空间中的动态扰动或流将合成源数据推向目标分布的角度解决了IGM问题。基于此方向，我们提出任意目标分布与源分布之间的分数差（SD）作为一种最优流，既能减少两者间的Kullback-Leibler散度，又能同时解决Schrödinger桥问题。我们将SD流应用于便捷的代理分布，这些代理分布的对齐性等价于原始分布的对齐性。我们证明了该形式在某些条件下与去噪扩散模型的形式等价。我们还表明，生成对抗网络的训练包含一个隐藏的数据优化子问题，在判别器最优且损失函数特定选择下，该子问题会诱导出SD流。因此，SD流为分别解决"生成建模三难困境"中三个挑战（高样本质量、模式覆盖和快速采样）的模型类之间提供了理论联系，从而为统一方法奠定了理论基础。