The tremendous progress in neural image generation, coupled with the emergence of seemingly omnipotent vision-language models has finally enabled text-based interfaces for creating and editing images. Handling generic images requires a diverse underlying generative model, hence the latest works utilize diffusion models, which were shown to surpass GANs in terms of diversity. One major drawback of diffusion models, however, is their relatively slow inference time. In this paper, we present an accelerated solution to the task of local text-driven editing of generic images, where the desired edits are confined to a user-provided mask. Our solution leverages a recent text-to-image Latent Diffusion Model (LDM), which speeds up diffusion by operating in a lower-dimensional latent space. We first convert the LDM into a local image editor by incorporating Blended Diffusion into it. Next we propose an optimization-based solution for the inherent inability of this LDM to accurately reconstruct images. Finally, we address the scenario of performing local edits using thin masks. We evaluate our method against the available baselines both qualitatively and quantitatively and demonstrate that in addition to being faster, our method achieves better precision than the baselines while mitigating some of their artifacts.

翻译：神经图像生成的巨大进步，加上看似无所不能的视觉语言模型的出现，最终实现了基于文本界面创建和编辑图像的功能。处理通用图像需要多样化的底层生成模型，因此最新研究采用扩散模型，这类模型在多样性方面已被证明超越生成对抗网络（GANs）。然而，扩散模型的一个主要缺点是推理时间相对较慢。本文针对通用图像的局部文本驱动编辑任务提出了一种加速解决方案，其中所需编辑区域限制在用户提供的掩膜内。我们的方案利用近期提出的文本到图像潜扩散模型（LDM），该模型通过在低维潜空间中运行来加速扩散过程。我们首先将混合扩散融入LDM，将其转化为局部图像编辑器。随后针对该LDM固有图像重建不准确的问题，提出基于优化的解决方案。最后，我们解决了使用薄掩膜执行局部编辑的场景。我们通过定性和定量方法将所提方法与现有基线进行对比，证明该方法在速度更快的同时，实现了比基线更高的精度，并减轻了它们的一些伪影问题。