Text-driven diffusion models have exhibited impressive generative capabilities, enabling various image editing tasks. In this paper, we propose TF-ICON, a novel Training-Free Image COmpositioN framework that harnesses the power of text-driven diffusion models for cross-domain image-guided composition. This task aims to seamlessly integrate user-provided objects into a specific visual context. Current diffusion-based methods often involve costly instance-based optimization or finetuning of pretrained models on customized datasets, which can potentially undermine their rich prior. In contrast, TF-ICON can leverage off-the-shelf diffusion models to perform cross-domain image-guided composition without requiring additional training, finetuning, or optimization. Moreover, we introduce the exceptional prompt, which contains no information, to facilitate text-driven diffusion models in accurately inverting real images into latent representations, forming the basis for compositing. Our experiments show that equipping Stable Diffusion with the exceptional prompt outperforms state-of-the-art inversion methods on various datasets (CelebA-HQ, COCO, and ImageNet), and that TF-ICON surpasses prior baselines in versatile visual domains. Code is available at https://github.com/Shilin-LU/TF-ICON

翻译：文本驱动的扩散模型展现了卓越的生成能力，可支持多种图像编辑任务。本文提出TF-ICON——一种新颖的免训练图像合成框架，利用文本驱动扩散模型实现跨域图像引导合成。该任务旨在将用户提供的物体无缝整合到特定视觉场景中。当前基于扩散模型的方法通常需要昂贵的实例级优化或在定制数据集上微调预训练模型，这种做法可能破坏其丰富的先验知识。相比之下，TF-ICON可直接利用现成扩散模型进行跨域图像引导合成，无需额外训练、微调或优化。此外，我们引入不含任何信息的"空提示"，使文本驱动扩散模型能够精确地将真实图像反演为潜在表示，为合成奠定基础。实验表明，为Stable Diffusion配备空提示后，在CelebA-HQ、COCO和ImageNet等多个数据集上均优于最先进的反演方法；TF-ICON在多种视觉领域中的表现也超越先前基线方法。代码已开源：https://github.com/Shilin-LU/TF-ICON