Implicit generative modeling (IGM) aims to produce samples of synthetic data matching the characteristics of a target data distribution. Recent work (e.g. score-matching networks, diffusion models) has approached the IGM problem from the perspective of pushing synthetic source data toward the target distribution via dynamical perturbations or flows in the ambient space. We introduce the score difference (SD) between arbitrary target and source distributions as a flow that optimally reduces the Kullback-Leibler divergence between them while also solving the Schr\"odinger bridge problem. We apply the SD flow to convenient proxy distributions, which are aligned if and only if the original distributions are aligned. We demonstrate the formal equivalence of this formulation to denoising diffusion models under certain conditions. However, unlike diffusion models, SD flow places no restrictions on the prior distribution. We also show that the training of generative adversarial networks includes a hidden data-optimization sub-problem, which induces the SD flow under certain choices of loss function when the discriminator is optimal. As a result, the SD flow provides a theoretical link between model classes that, taken together, address all three challenges of the "generative modeling trilemma": high sample quality, mode coverage, and fast sampling.

翻译：隐式生成建模旨在生成与目标数据分布特征相匹配的合成数据样本。近期研究（例如分数匹配网络、扩散模型）从通过动态扰动或环境空间中的流推动合成源数据向目标分布迁移的角度探讨了隐式生成建模问题。我们引入任意目标分布与源分布之间的分数差值，将其作为能够最优减小两者间KL散度并同时求解薛定谔桥问题的流。将该分数差值流应用于便捷代理分布，这些代理分布的对齐性等价于原始分布的对齐性。我们证明该框架在特定条件下与去噪扩散模型形式等价。但与扩散模型不同的是，分数差值流对先验分布不作任何限制。我们还表明生成对抗网络的训练包含一个隐藏的数据优化子问题，当判别器最优时，该子问题在特定损失函数选择下可诱导出分数差值流。因此，分数差值流为不同模型类别间建立了理论联系，这些模型共同应对"生成建模三难困境"的三大挑战：高样本质量、模式覆盖与快速采样。