Text-driven diffusion models have exhibited impressive generative capabilities, enabling various image editing tasks. In this paper, we propose TF-ICON, a novel Training-Free Image COmpositioN framework that harnesses the power of text-driven diffusion models for cross-domain image-guided composition. This task aims to seamlessly integrate user-provided objects into a specific visual context. Current diffusion-based methods often involve costly instance-based optimization or finetuning of pretrained models on customized datasets, which can potentially undermine their rich prior. In contrast, TF-ICON can leverage off-the-shelf diffusion models to perform cross-domain image-guided composition without requiring additional training, finetuning, or optimization. Moreover, we introduce the exceptional prompt, which contains no information, to facilitate text-driven diffusion models in accurately inverting real images into latent representations, forming the basis for compositing. Our experiments show that equipping Stable Diffusion with the exceptional prompt outperforms state-of-the-art inversion methods on various datasets (CelebA-HQ, COCO, and ImageNet), and that TF-ICON surpasses prior baselines in versatile visual domains. Code is available at https://github.com/Shilin-LU/TF-ICON

翻译：文本驱动的扩散模型在生成能力上展现出卓越表现，可支持多种图像编辑任务。本文提出TF-ICON——一种新型无训练图像合成框架，利用文本驱动扩散模型实现跨域图像引导合成。该任务旨在将用户提供的物体无缝融入特定视觉语境中。当前基于扩散模型的方法常涉及昂贵的实例级优化或对预训练模型在定制数据集上的微调，这可能会削弱其丰富的先验知识。相比之下，TF-ICON无需额外训练、微调或优化，即可直接利用现成扩散模型执行跨域图像引导合成。此外，我们引入不含任何信息的“空白提示”（exceptional prompt），辅助文本驱动扩散模型精准地将真实图像反转为潜在表示，为合成奠定基础。实验表明，将Stable Diffusion与空白提示结合，在CelebA-HQ、COCO和ImageNet等多个数据集上优于最先进的反演方法；而TF-ICON在多种视觉域中亦超越先前基线方法。代码已开源：https://github.com/Shilin-LU/TF-ICON