Typical generative diffusion models rely on a Gaussian diffusion process for training the backward transformations, which can then be used to generate samples from Gaussian noise. However, real world data often takes place in discrete-state spaces, including many scientific applications. Here, we develop a theoretical formulation for arbitrary discrete-state Markov processes in the forward diffusion process using exact (as opposed to variational) analysis. We relate the theory to the existing continuous-state Gaussian diffusion as well as other approaches to discrete diffusion, and identify the corresponding reverse-time stochastic process and score function in the continuous-time setting, and the reverse-time mapping in the discrete-time setting. As an example of this framework, we introduce ``Blackout Diffusion'', which learns to produce samples from an empty image instead of from noise. Numerical experiments on the CIFAR-10, Binarized MNIST, and CelebA datasets confirm the feasibility of our approach. Generalizing from specific (Gaussian) forward processes to discrete-state processes without a variational approximation sheds light on how to interpret diffusion models, which we discuss.

翻译：典型的生成扩散模型依赖于高斯扩散过程来训练反向变换，从而能够从高斯噪声中生成样本。然而，现实世界的数据往往存在于离散状态空间中，包括许多科学应用场景。在此，我们采用精确（而非变分）分析方法，为前向扩散过程中的任意离散状态马尔可夫过程建立了理论框架。我们将该理论与现有的连续状态高斯扩散及其他离散扩散方法相关联，确定了连续时间设置下对应的反向时间随机过程与得分函数，以及离散时间设置下的反向时间映射。作为该框架的实例，我们引入了“黑出扩散”方法，该方法学会从空图像而非噪声中生成样本。在CIFAR-10、二值化MNIST和CelebA数据集上的数值实验证实了我们方法的可行性。从特定的（高斯）前向过程推广到无需变分近似的离散状态过程，为理解扩散模型的解释方式提供了新视角，我们对此进行了讨论。