This work studies discrete diffusion probabilistic models with applications to natural language generation. We derive an alternative yet equivalent formulation of the sampling from discrete diffusion processes and leverage this insight to develop a family of reparameterized discrete diffusion models. The derived generic framework is highly flexible, offers a fresh perspective of the generation process in discrete diffusion models, and features more effective training and decoding techniques. We conduct extensive experiments to evaluate the text generation capability of our model, demonstrating significant improvements over existing diffusion models.

翻译：本研究探讨了离散扩散概率模型及其在自然语言生成中的应用。我们从离散扩散过程中推导出一种等价但形式不同的采样公式，并基于此提出了一类重参数化的离散扩散模型。所提出的通用框架具有高度灵活性，为离散扩散模型的生成过程提供了全新视角，并支持更高效的训练与解码技术。通过大量实验评估该模型的文本生成能力，结果表明其相比现有扩散模型取得了显著改进。