Implicit generative modeling (IGM) aims to produce samples of synthetic data matching the characteristics of a target data distribution. Recent work (e.g. score-matching networks, diffusion models) has approached the IGM problem from the perspective of pushing synthetic source data toward the target distribution via dynamical perturbations or flows in the ambient space. In this direction, we present the score difference (SD) between arbitrary target and source distributions as a flow that optimally reduces the Kullback-Leibler divergence between them. We apply the SD flow to convenient proxy distributions, which are aligned if and only if the original distributions are aligned. We demonstrate the formal equivalence of this formulation to denoising diffusion models under certain conditions. We also show that the training of generative adversarial networks includes a hidden data-optimization sub-problem, which induces the SD flow under certain choices of loss function when the discriminator is optimal. As a result, the SD flow provides a theoretical link between model classes that individually address the three challenges of the "generative modeling trilemma" -- high sample quality, mode coverage, and fast sampling -- thereby setting the stage for a unified approach.

翻译：隐式生成建模旨在生成与目标数据分布特征匹配的合成数据样本。近期研究（如分数匹配网络、扩散模型）从动态扰动或环境空间流的角度推动合成源数据向目标分布逼近。在此方向上，我们提出任意目标分布与源分布之间的分数差可构建为一种能最优降低两者间KL散度的流。我们将该分数差流应用于便捷的代理分布，这些代理分布当且仅当原始分布对齐时才会对齐。我们证明了该形式化描述在特定条件下与去噪扩散模型的等价性。同时揭示了生成对抗网络的训练包含一个隐含的数据优化子问题，当判别器达到最优时，该问题在特定损失函数选择下会诱导出分数差流。因此，分数差流为分别应对“生成建模三难困境”——高样本质量、模式覆盖与快速采样——的各类模型建立了理论联系，从而为统一方法奠定了基础。