This work studies discrete diffusion probabilistic models with applications to natural language generation. We derive an alternative yet equivalent formulation of the sampling from discrete diffusion processes and leverage this insight to develop a family of reparameterized discrete diffusion models. The derived generic framework is highly flexible, offers a fresh perspective of the generation process in discrete diffusion models, and features more effective training and decoding techniques. We conduct extensive experiments to evaluate the text generation capability of our model, demonstrating significant improvements over existing diffusion models.

翻译：本研究探讨了离散扩散概率模型在自然语言生成中的应用。我们推导出离散扩散过程采样的另一种等价形式，并利用这一洞见开发了重参数化离散扩散模型系列。所提出的通用框架具有高度灵活性，为离散扩散模型的生成过程提供了全新视角，并采用了更高效的训练与解码技术。我们通过大量实验评估了模型在文本生成方面的能力，结果表明其性能显著优于现有扩散模型。