Recently, diffusion models have emerged as a new paradigm for generative models. Despite the success in domains using continuous signals such as vision and audio, adapting diffusion models to natural language is under-explored due to the discrete nature of texts, especially for conditional generation. We tackle this challenge by proposing DiffuSeq: a diffusion model designed for sequence-to-sequence (Seq2Seq) text generation tasks. Upon extensive evaluation over a wide range of Seq2Seq tasks, we find DiffuSeq achieving comparable or even better performance than six established baselines, including a state-of-the-art model that is based on pre-trained language models. Apart from quality, an intriguing property of DiffuSeq is its high diversity during generation, which is desired in many Seq2Seq tasks. We further include a theoretical analysis revealing the connection between DiffuSeq and autoregressive/non-autoregressive models. Bringing together theoretical analysis and empirical evidence, we demonstrate the great potential of diffusion models in complex conditional language generation tasks. Code is available at \url{https://github.com/Shark-NLP/DiffuSeq}

翻译：近年来，扩散模型已成为生成模型的新范式。尽管在视觉和音频等连续信号领域取得了成功，但由于文本的离散特性，扩散模型在自然语言领域的应用，特别是在条件生成任务中，仍处于探索不足的状态。我们提出DiffuSeq这一专为序列到序列（Seq2Seq）文本生成任务设计的扩散模型，以应对这一挑战。通过在广泛的Seq2Seq任务上进行全面评估，我们发现DiffuSeq在性能上可与六种现有基准模型（包括基于预训练语言模型的最先进方法）媲美甚至更优。除生成质量外，DiffuSeq的一个显著特性是其生成过程具有高多样性，这在许多Seq2Seq任务中具有重要价值。我们进一步通过理论分析揭示了DiffuSeq与自回归/非自回归模型之间的关联。结合理论分析与实验证据，我们证明了扩散模型在复杂条件语言生成任务中的巨大潜力。代码已开源：\url{https://github.com/Shark-NLP/DiffuSeq}