Diffusion models have garnered considerable interest in the field of text generation. Several studies have explored text diffusion models with different structures and applied them to various tasks, including named entity recognition and summarization. However, there exists a notable disparity between the "easy-first" text generation process of current diffusion models and the "keyword-first" natural text generation process of humans, which has received limited attention. To bridge this gap, we propose InfoDiffusion, a non-autoregressive text diffusion model. Our approach introduces a "keyinfo-first" generation strategy and incorporates a noise schedule based on the amount of text information. In addition, InfoDiffusion combines self-conditioning with a newly proposed partially noising model structure. Experimental results show that InfoDiffusion outperforms the baseline model in terms of generation quality and diversity, as well as exhibiting higher sampling efficiency.

翻译：扩散模型在文本生成领域引起了广泛关注。多项研究探索了具有不同结构的文本扩散模型，并将其应用于命名实体识别和摘要生成等各类任务。然而，当前扩散模型的"先易后难"文本生成过程与人类"关键词优先"的自然文本生成过程之间存在显著差异，这一问题尚未得到充分关注。为弥合这一差距，我们提出了InfoDiffusion——一种非自回归文本扩散模型。该方法采用"关键信息优先"的生成策略，并基于文本信息量引入了噪声调度机制。此外，InfoDiffusion将自条件化与新颖的部分加噪模型结构相结合。实验结果表明，InfoDiffusion在生成质量和多样性方面均优于基线模型，同时展现出更高的采样效率。