Diffusion probabilistic models have shown great success in generating high-quality images controllably, and researchers have tried to utilize this controllability into text generation domain. Previous works on diffusion-based language models have shown that they can be trained without external knowledge (such as pre-trained weights) and still achieve stable performance and controllability. In this paper, we trained a diffusion-based model on StylePTB dataset, the standard benchmark for fine-grained text style transfers. The tasks in StylePTB requires much more refined control over the output text compared to tasks evaluated in previous works, and our model was able to achieve state-of-the-art performance on StylePTB on both individual and compositional transfers. Moreover, our model, trained on limited data from StylePTB without external knowledge, outperforms previous works that utilized pretrained weights, embeddings, and external grammar parsers, and this may indicate that diffusion-based language models have great potential under low-resource settings.

翻译：扩散概率模型在可控生成高质量图像方面取得了显著成功，研究者们尝试将这种可控性应用于文本生成领域。先前关于基于扩散的语言模型的研究表明，这些模型无需外部知识（如预训练权重）即可训练，且能实现稳定的性能和可控性。本文在StylePTB数据集（细粒度文本风格迁移的标准基准）上训练了一个基于扩散的模型。与先前研究评估的任务相比，StylePTB中的任务需要对输出文本进行更精细的控制。我们的模型在StylePTB的单一迁移和组合迁移任务上均达到了最先进的性能。此外，该模型仅使用StylePTB的有限数据训练，无需外部知识，即优于采用预训练权重、嵌入和外部语法解析器的先前工作。这或许表明基于扩散的语言模型在低资源场景下具有巨大潜力。