Uniform-state discrete diffusion models hold the promise of fast text generation due to their inherent ability to self-correct. However, they are typically outperformed by autoregressive models and masked diffusion models. In this work, we narrow this performance gap by leveraging a key insight: Uniform-state diffusion processes naturally emerge from an underlying Gaussian diffusion. Our method, Duo, transfers powerful techniques from Gaussian diffusion to improve both training and sampling. First, we introduce a curriculum learning strategy guided by the Gaussian process, doubling training speed by reducing variance. Models trained with curriculum learning surpass autoregressive models in zero-shot perplexity on 3 of 7 benchmarks. Second, we present Discrete Consistency Distillation, which adapts consistency distillation from the continuous to the discrete setting. This algorithm unlocks few-step generation in diffusion language models by accelerating sampling by two orders of magnitude. We provide the code, model checkpoints, and video tutorials on the project page: http://s-sahoo.github.io/duo

翻译：均匀态离散扩散模型因其固有的自校正能力，在快速文本生成方面展现出潜力。然而，其性能通常落后于自回归模型与掩码扩散模型。本研究通过揭示一个关键机制来缩小这一性能差距：均匀态扩散过程本质上源于底层的高斯扩散。我们提出的Duo方法，将高斯扩散中的强大技术迁移至离散领域，从而同时改进训练与采样过程。首先，我们引入一种由高斯过程引导的课程学习策略，通过降低方差使训练速度提升一倍。采用课程学习训练的模型在7项基准测试中的3项上，其零样本困惑度超越了自回归模型。其次，我们提出了离散一致性蒸馏算法，将连续域的一致性蒸馏技术适配至离散场景。该算法通过将采样速度提升两个数量级，实现了扩散语言模型中的少步生成。相关代码、模型检查点及视频教程已发布于项目页面：http://s-sahoo.github.io/duo