This paper attempts to present the stochastic differential equations of diffusion models in a manner that is accessible to a broad audience. The diffusion process is defined over a population density in R^d. Of particular interest is a population of images. In a diffusion model one first defines a diffusion process that takes a sample from the population and gradually adds noise until only noise remains. The fundamental idea is to sample from the population by a reverse-diffusion process mapping pure noise to a population sample. The diffusion process is defined independent of any ``interpretation'' but can be analyzed using the mathematics of variational auto-encoders (the ``VAE interpretation'') or the Fokker-Planck equation (the ``score-matching intgerpretation''). Both analyses yield reverse-diffusion methods involving the score function. The Fokker-Planck analysis yields a family of reverse-diffusion SDEs parameterized by any desired level of reverse-diffusion noise including zero (deterministic reverse-diffusion). The VAE analysis yields the reverse-diffusion SDE at the same noise level as the diffusion SDE. The VAE analysis also yields a useful expression for computing the population probabilities of a given point (image). This formula for the probability of a given point does not seem to follow naturally from the Fokker-Planck analysis. Much, but apparently not all, of the mathematics presented here can be found in the literature. Attributions are given at the end of the paper.
翻译:本文试图以易于广大读者理解的方式阐述扩散模型的随机微分方程。扩散过程定义在R^d空间中的群体密度上,特别关注图像群体。在扩散模型中,首先定义一个扩散过程,该过程从群体中采集样本并逐渐添加噪声,直至仅剩噪声。核心思想是通过逆扩散过程实现群体采样,将纯噪声映射为群体样本。扩散过程的定义独立于任何“解释”,但可通过变分自编码器的数学方法(“VAE解释”)或福克-普朗克方程(“分数匹配解释”)进行分析。两种分析均得出涉及分数函数的逆扩散方法。福克-普朗克分析导出一族由任意期望逆扩散噪声水平参数化的逆扩散随机微分方程,包括零噪声(确定性逆扩散)。VAE分析得出与扩散随机微分方程相同噪声水平的逆扩散随机微分方程。VAE分析还提供了一种计算给定点(图像)群体概率的有用表达式,该公式似乎无法自然地从福克-普朗克分析中得出。本文介绍的数学内容大部分(但并非全部)可从现有文献中找到。文末附有引用来源。