Diffusion models gain increasing popularity for their generative capabilities. Recently, there have been surging needs to generate customized images by inverting diffusion models from exemplar images. However, existing inversion methods mainly focus on capturing object appearances. How to invert object relations, another important pillar in the visual world, remains unexplored. In this work, we propose ReVersion for the Relation Inversion task, which aims to learn a specific relation (represented as "relation prompt") from exemplar images. Specifically, we learn a relation prompt from a frozen pre-trained text-to-image diffusion model. The learned relation prompt can then be applied to generate relation-specific images with new objects, backgrounds, and styles. Our key insight is the "preposition prior" - real-world relation prompts can be sparsely activated upon a set of basis prepositional words. Specifically, we propose a novel relation-steering contrastive learning scheme to impose two critical properties of the relation prompt: 1) The relation prompt should capture the interaction between objects, enforced by the preposition prior. 2) The relation prompt should be disentangled away from object appearances. We further devise relation-focal importance sampling to emphasize high-level interactions over low-level appearances (e.g., texture, color). To comprehensively evaluate this new task, we contribute ReVersion Benchmark, which provides various exemplar images with diverse relations. Extensive experiments validate the superiority of our approach over existing methods across a wide range of visual relations.

翻译：扩散模型因其生成能力而日益流行。近期，通过从示例图像中反演扩散模型来生成定制化图像的需求激增。然而，现有的反演方法主要聚焦于捕捉物体外观。如何在视觉世界的另一重要支柱——物体关系上进行反演，仍未被探索。本文提出ReVersion用于关系反演任务，旨在从示例图像中学习特定关系（表示为“关系提示”）。具体而言，我们从冻结的预训练文本到图像扩散模型中学习关系提示。学习到的关系提示随后可应用于生成具有新物体、背景和风格的指定关系图像。我们的核心洞察是“介词先验”——现实世界的关系提示可以在一组基础介词词汇上稀疏激活。具体而言，我们提出了一种新颖的关系导向对比学习方案，以赋予关系提示两个关键属性：1）关系提示应捕捉物体之间的交互，由介词先验强制实现；2）关系提示应与物体外观解耦。我们还设计了关系焦点重要性采样，以强调高层交互而非低层外观（如纹理、颜色）。为全面评估这一新任务，我们贡献了ReVersion基准测试，提供了多种包含不同关系的示例图像。大量实验验证了我们的方法在多种视觉关系上相较于现有方法的优越性。