Text-to-Image synthesis is the task of generating an image according to a specific text description. Generative Adversarial Networks have been considered the standard method for image synthesis virtually since their introduction; today, Denoising Diffusion Probabilistic Models are recently setting a new baseline, with remarkable results in Text-to-Image synthesis, among other fields. Aside its usefulness per se, it can also be particularly relevant as a tool for data augmentation to aid training models for other document image processing tasks. In this work, we present a latent diffusion-based method for styled text-to-text-content-image generation on word-level. Our proposed method manages to generate realistic word image samples from different writer styles, by using class index styles and text content prompts without the need of adversarial training, writer recognition, or text recognition. We gauge system performance with Frechet Inception Distance, writer recognition accuracy, and writer retrieval. We show that the proposed model produces samples that are aesthetically pleasing, help boosting text recognition performance, and gets similar writer retrieval score as real data.

翻译：文本到图像合成是指根据特定文本描述生成图像的任务。生成对抗网络自提出以来一直被视为图像合成的标准方法；而如今，去噪扩散概率模型正在这一领域及其他领域（包括文本到图像合成）中确立新的基准。除了其本身的实用性外，该方法作为数据增强工具，在辅助训练其他文档图像处理任务模型方面也具有特别重要的意义。本文提出了一种基于潜在扩散的词级风格化文本内容图像生成方法。所提出的方法通过使用类别索引风格和文本内容提示，无需对抗训练、书写者识别或文本识别，即可生成来自不同书写者风格的真实单词图像样本。我们采用弗雷歇初始距离、书写者识别准确率和书写者检索来评估系统性能。结果表明，所提出的模型生成的样本具有美学吸引力，有助于提升文本识别性能，并获得了与真实数据相似的书写者检索得分。