Large-scale Text-to-Image (T2I) diffusion models demonstrate significant generation capabilities based on textual prompts. Based on the T2I diffusion models, text-guided image editing research aims to empower users to manipulate generated images by altering the text prompts. However, existing image editing techniques are prone to editing over unintentional regions that are beyond the intended target area, primarily due to inaccuracies in cross-attention maps. To address this problem, we propose Localization-aware Inversion (LocInv), which exploits segmentation maps or bounding boxes as extra localization priors to refine the cross-attention maps in the denoising phases of the diffusion process. Through the dynamic updating of tokens corresponding to noun words in the textual input, we are compelling the cross-attention maps to closely align with the correct noun and adjective words in the text prompt. Based on this technique, we achieve fine-grained image editing over particular objects while preventing undesired changes to other regions. Our method LocInv, based on the publicly available Stable Diffusion, is extensively evaluated on a subset of the COCO dataset, and consistently obtains superior results both quantitatively and qualitatively.The code will be released at https://github.com/wangkai930418/DPL

翻译：大规模文本到图像（T2I）扩散模型基于文本提示展现出显著的生成能力。基于T2I扩散模型，文本引导图像编辑研究旨在使用户能够通过修改文本提示来操控生成的图像。然而，现有图像编辑技术容易在预期目标区域之外的无关区域产生编辑，这主要是由于交叉注意力图的不准确性所致。为解决这一问题，我们提出定位感知反演（LocInv），该方法利用分割图或边界框作为额外的定位先验，以精炼扩散过程去噪阶段中的交叉注意力图。通过对文本输入中名词对应的词元进行动态更新，我们迫使交叉注意力图与文本提示中的正确名词和形容词紧密对齐。基于该技术，我们实现了对特定目标的细粒度图像编辑，同时防止对其他区域产生非期望的修改。我们的方法LocInv基于公开可用的Stable Diffusion，在COCO数据集的一个子集上进行了广泛评估，并在定量和定性结果上均持续取得优越表现。代码将发布于 https://github.com/wangkai930418/DPL