Diffusion model based language-guided image editing has achieved great success recently. However, existing state-of-the-art diffusion models struggle with rendering correct text and text style during generation. To tackle this problem, we propose a universal self-supervised text editing diffusion model (DiffUTE), which aims to replace or modify words in the source image with another one while maintaining its realistic appearance. Specifically, we build our model on a diffusion model and carefully modify the network structure to enable the model for drawing multilingual characters with the help of glyph and position information. Moreover, we design a self-supervised learning framework to leverage large amounts of web data to improve the representation ability of the model. Experimental results show that our method achieves an impressive performance and enables controllable editing on in-the-wild images with high fidelity. Our code will be avaliable in \url{https://github.com/chenhaoxing/DiffUTE}.

翻译：摘要：基于扩散模型的文本引导图像编辑近期取得了巨大成功。然而，现有最先进的扩散模型在生成过程中难以正确渲染文本及文本风格。为解决这一问题，我们提出了一种通用的自监督文本编辑扩散模型（DiffUTE），旨在替换或修改源图像中的词语，同时保持其真实外观。具体而言，我们以扩散模型为基础构建模型，并精心修改网络结构，借助字形和位置信息使模型能够绘制多语言字符。此外，我们设计了一个自监督学习框架，利用大量网络数据提升模型的表征能力。实验结果表明，所提方法在野外图像上实现了令人瞩目的可控编辑效果，并保持了高度保真度。我们的代码将在 \url{https://github.com/chenhaoxing/DiffUTE} 中提供。