Text-conditioned image editing has recently attracted considerable interest. However, most methods are currently either limited to specific editing types (e.g., object overlay, style transfer), or apply to synthetically generated images, or require multiple input images of a common object. In this paper we demonstrate, for the very first time, the ability to apply complex (e.g., non-rigid) text-guided semantic edits to a single real image. For example, we can change the posture and composition of one or multiple objects inside an image, while preserving its original characteristics. Our method can make a standing dog sit down or jump, cause a bird to spread its wings, etc. -- each within its single high-resolution natural image provided by the user. Contrary to previous work, our proposed method requires only a single input image and a target text (the desired edit). It operates on real images, and does not require any additional inputs (such as image masks or additional views of the object). Our method, which we call "Imagic", leverages a pre-trained text-to-image diffusion model for this task. It produces a text embedding that aligns with both the input image and the target text, while fine-tuning the diffusion model to capture the image-specific appearance. We demonstrate the quality and versatility of our method on numerous inputs from various domains, showcasing a plethora of high quality complex semantic image edits, all within a single unified framework.

翻译：摘要：基于文本条件的图像编辑近期引起了广泛关注。然而，现有方法大多局限于特定编辑类型（如物体叠加、风格迁移），或仅适用于合成生成的图像，亦或需要同一物体的多张输入图像。本文首次证明了将复杂（例如非刚性）的文本引导语义编辑应用于单张真实图像的能力。例如，我们可以改变图像中一个或多个物体的姿态与构图，同时保留其原始特征。我们的方法能让站立的狗坐下、跳跃，使鸟儿展翅等——每种编辑均在用户提供的单张高分辨率自然图像上完成。与以往工作不同，本文方法仅需单张输入图像和目标文本（期望编辑内容）。它直接作用于真实图像，无需任何额外输入（如图像掩码或物体的其他视角）。我们提出的方法名为“Imagic”，利用预训练的文本到图像扩散模型来完成该任务。该方法生成与输入图像及目标文本均对齐的文本嵌入，同时微调扩散模型以捕捉图像特有的外观。我们在多个领域的广泛输入上展示了方法的质性与多样性，所有编辑均通过统一框架实现，涵盖了大量高质量、复杂的语义图像编辑案例。