We focus on the fundamental mathematical structure of score-based generative models (SGMs). We first formulate SGMs in terms of the Wasserstein proximal operator (WPO) and demonstrate that, via mean-field games (MFGs), the WPO formulation reveals mathematical structure that describes the inductive bias of diffusion and score-based models. In particular, MFGs yield optimality conditions in the form of a pair of coupled partial differential equations: a forward-controlled Fokker-Planck (FP) equation, and a backward Hamilton-Jacobi-Bellman (HJB) equation. Via a Cole-Hopf transformation and taking advantage of the fact that the cross-entropy can be related to a linear functional of the density, we show that the HJB equation is an uncontrolled FP equation. Second, with the mathematical structure at hand, we present an interpretable kernel-based model for the score function which dramatically improves the performance of SGMs in terms of training samples and training time. In addition, the WPO-informed kernel model is explicitly constructed to avoid the recently studied memorization effects of score-based generative models. The mathematical form of the new kernel-based models in combination with the use of the terminal condition of the MFG reveals new explanations for the manifold learning and generalization properties of SGMs, and provides a resolution to their memorization effects. Finally, our mathematically informed, interpretable kernel-based model suggests new scalable bespoke neural network architectures for high-dimensional applications.

翻译：我们聚焦于基于分数的生成模型（SGMs）的基本数学结构。首先，我们通过Wasserstein邻近算子（WPO）对SGMs进行公式化，并借助平均场博弈（MFGs）证明，WPO公式化揭示了描述扩散模型与基于分数模型归纳偏置的数学结构。具体而言，MFG导出了一对耦合偏微分方程形式的最优性条件：前向受控Fokker-Planck（FP）方程和后向Hamilton-Jacobi-Bellman（HJB）方程。通过Cole-Hopf变换，并利用交叉熵可与密度的线性泛函相关联这一事实，我们证明了HJB方程为非受控FP方程。其次，基于所获得的数学结构，我们提出了一种可解释的核函数分数模型，该模型在训练样本量和训练时间方面显著提升了SGMs的性能。此外，该基于WPO的核模型被显式构建，以避免近期研究中发现的基于分数的生成模型的记忆效应。新核模型的数学形式与MFG终端条件的结合，揭示了SGMs流形学习与泛化特性的新解释，并为其记忆效应提供了解决方案。最后，我们的数学驱动、可解释的核模型为高维应用提出了可扩展的定制化神经网络架构。