Discrete diffusion models, like continuous diffusion models, generate high-quality samples by gradually undoing noise applied to datapoints with a Markov process. Gradual generation in theory comes with many conceptual benefits; for example, inductive biases can be incorporated into the noising Markov process, and access to improved sampling algorithms. In practice, however, the consistently best performing discrete diffusion model is, surprisingly, masking diffusion, which does not denoise gradually. Here we explain the superior performance of masking diffusion by noting that it makes use of a fundamental difference between continuous and discrete Markov processes: discrete Markov processes evolve by discontinuous jumps at a fixed rate and, unlike other discrete diffusion models, masking diffusion builds in the known distribution of jump times and only learns where to jump to. We show that we can similarly bake in the known distribution of jump times into any discrete diffusion model. The resulting models - schedule-conditioned discrete diffusion (SCUD) - generalize classical discrete diffusion and masking diffusion. By applying SCUD to models with noising processes that incorporate inductive biases on images, text, and protein data, we build models that outperform masking.

翻译：与连续扩散模型类似，离散扩散模型通过马尔可夫过程逐步去除数据点所添加的噪声来生成高质量样本。理论上，渐进式生成具有诸多概念优势；例如，归纳偏置可融入噪声化马尔可夫过程，并能采用更优的采样算法。然而实践中，表现始终最佳的离散扩散模型却是掩码扩散——这令人惊讶，因为它并非渐进去噪。本文通过指出掩码扩散利用了连续与离散马尔可夫过程的根本差异来解释其优越性能：离散马尔可夫过程以固定速率通过不连续跳跃演化，而与其他离散扩散模型不同，掩码扩散内置了已知的跳跃时间分布，仅需学习跳跃目标位置。我们证明可将同样的跳跃时间分布特性融入任意离散扩散模型。由此得到的模型——时间表条件化离散扩散（SCUD）——推广了经典离散扩散与掩码扩散。通过将SCUD应用于在图像、文本和蛋白质数据上融入归纳偏置的噪声化过程模型，我们构建了超越掩码扩散性能的新模型。