Recently large-scale language-image models (e.g., text-guided diffusion models) have considerably improved the image generation capabilities to generate photorealistic images in various domains. Based on this success, current image editing methods use texts to achieve intuitive and versatile modification of images. To edit a real image using diffusion models, one must first invert the image to a noisy latent from which an edited image is sampled with a target text prompt. However, most methods lack one of the following: user-friendliness (e.g., additional masks or precise descriptions of the input image are required), generalization to larger domains, or high fidelity to the input image. In this paper, we design an accurate and quick inversion technique, Prompt Tuning Inversion, for text-driven image editing. Specifically, our proposed editing method consists of a reconstruction stage and an editing stage. In the first stage, we encode the information of the input image into a learnable conditional embedding via Prompt Tuning Inversion. In the second stage, we apply classifier-free guidance to sample the edited image, where the conditional embedding is calculated by linearly interpolating between the target embedding and the optimized one obtained in the first stage. This technique ensures a superior trade-off between editability and high fidelity to the input image of our method. For example, we can change the color of a specific object while preserving its original shape and background under the guidance of only a target text prompt. Extensive experiments on ImageNet demonstrate the superior editing performance of our method compared to the state-of-the-art baselines.

翻译：最近，大规模语言-图像模型（如文本引导扩散模型）显著提升了图像生成能力，能够在各个领域生成逼真的图像。基于这一成功，当前的图像编辑方法利用文本实现对图像的直观且多功能的修改。为了使用扩散模型编辑真实图像，首先需要将图像反演为一个含噪潜变量，然后根据目标文本提示从中采样得到编辑后的图像。然而，大多数方法在以下方面存在不足：用户友好性（例如，需要额外的掩码或输入图像的精确描述）、对更大领域的泛化能力，或对输入图像的高保真度。在本文中，我们设计了一种精确且快速的反演技术——提示调谐反演，用于文本驱动的图像编辑。具体来说，我们提出的编辑方法包括一个重建阶段和一个编辑阶段。在第一阶段，我们通过提示调谐反演将输入图像的信息编码为可学习的条件嵌入。在第二阶段，我们应用无分类器引导来采样编辑后的图像，其中条件嵌入通过线性插值计算目标嵌入与第一阶段获得的优化嵌入得到。该技术确保了我们的方法在可编辑性与对输入图像的高保真度之间达到优越的权衡。例如，我们可以在仅通过目标文本提示的引导下，改变特定对象的颜色，同时保留其原始形状和背景。在ImageNet上的大量实验表明，与最先进的基线方法相比，我们的方法具有卓越的编辑性能。