Recently, there has been a growing interest in developing diffusion-based text-to-image generative models capable of generating coherent and well-formed visual text. In this paper, we propose a novel and efficient approach called GlyphControl to address this task. Unlike existing methods that rely on character-aware text encoders like ByT5 and require retraining of text-to-image models, our approach leverages additional glyph conditional information to enhance the performance of the off-the-shelf Stable-Diffusion model in generating accurate visual text. By incorporating glyph instructions, users can customize the content, location, and size of the generated text according to their specific requirements. To facilitate further research in visual text generation, we construct a training benchmark dataset called LAION-Glyph. We evaluate the effectiveness of our approach by measuring OCR-based metrics and CLIP scores of the generated visual text. Our empirical evaluations demonstrate that GlyphControl outperforms the recent DeepFloyd IF approach in terms of OCR accuracy and CLIP scores, highlighting the efficacy of our method.

翻译：近年来，基于扩散的文本到图像生成模型在生成连贯且形态良好的视觉文本方面引起了广泛关注。本文提出了一种名为GlyphControl的新型高效方法来解决该任务。与现有依赖字符感知文本编码器（如ByT5）且需要重新训练文本到图像模型的方法不同，我们的方法利用额外的字形条件信息来提升现成的稳定扩散模型（Stable-Diffusion）在生成准确视觉文本方面的性能。通过引入字形指令，用户可以根据具体需求自定义所生成文本的内容、位置和尺寸。为促进视觉文本生成的进一步研究，我们构建了一个名为LAION-Glyph的训练基准数据集。我们通过测量生成视觉文本的OCR指标和CLIP分数来评估方法的有效性。实证评估表明，GlyphControl在OCR准确率和CLIP分数上均优于最新的DeepFloyd IF方法，突显了我们方法的有效性。