Non-autoregressive (NAR) text generation has attracted much attention in the field of natural language processing, which greatly reduces the inference latency but has to sacrifice the generation accuracy. Recently, diffusion models, a class of latent variable generative models, have been introduced into NAR text generation, showing improved generation quality. In this survey, we review the recent progress in diffusion models for NAR text generation. As the background, we first present the general definition of diffusion models and the text diffusion models, and then discuss their merits for NAR generation. As the core content, we further introduce two mainstream diffusion models in existing text diffusion works, and review the key designs of the diffusion process. Moreover, we discuss the utilization of pre-trained language models (PLMs) for text diffusion models and introduce optimization techniques for text data. Finally, we discuss several promising directions and conclude this paper. Our survey aims to provide researchers with a systematic reference of related research on text diffusion models for NAR generation.

翻译：非自回归文本生成在自然语言处理领域引起了广泛关注，该方法显著降低了推理延迟，但不得不牺牲生成精度。近年来，作为一类潜变量生成模型的扩散模型被引入非自回归文本生成，展现出改进的生成质量。本综述回顾了扩散模型在非自回归文本生成中的最新进展。作为背景，我们首先介绍扩散模型及文本扩散模型的一般定义，随后讨论其对非自回归生成的优势。作为核心内容，我们进一步介绍现有文本扩散工作中的两种主流扩散模型，并评述扩散过程的关键设计。此外，我们探讨预训练语言模型在文本扩散模型中的应用，并介绍针对文本数据的优化技术。最后，我们讨论若干有前景的研究方向并总结全文。本综述旨在为研究人员提供关于非自回归生成文本扩散模型相关研究的系统性参考。