Composed Image Retrieval (CIR) aims to retrieve a target image based on a query composed of a reference image and a relative caption that describes the difference between the two images. The high effort and cost required for labeling datasets for CIR hamper the widespread usage of existing methods, as they rely on supervised learning. In this work, we propose a new task, Zero-Shot CIR (ZS-CIR), that aims to address CIR without requiring a labeled training dataset. Our approach, named zero-Shot composEd imAge Retrieval with textuaL invErsion (SEARLE), maps the visual features of the reference image into a pseudo-word token in CLIP token embedding space and integrates it with the relative caption. To support research on ZS-CIR, we introduce an open-domain benchmarking dataset named Composed Image Retrieval on Common Objects in context (CIRCO), which is the first dataset for CIR containing multiple ground truths for each query. The experiments show that SEARLE exhibits better performance than the baselines on the two main datasets for CIR tasks, FashionIQ and CIRR, and on the proposed CIRCO. The dataset, the code and the model are publicly available at https://github.com/miccunifi/SEARLE .

翻译：组合图像检索旨在基于由参考图像和描述两图差异的相对说明组成的查询来检索目标图像。由于现有方法依赖监督学习，为组合图像检索标注数据集所需的高昂人力与成本阻碍了其广泛应用。本文提出一项新任务——零样本组合图像检索，目标是在无需标记训练数据集的情况下解决组合图像检索问题。我们提出的方法——基于文本反转的零样本组合图像检索，将参考图像的视觉特征映射为CLIP令牌嵌入空间中的伪词令牌，并将其与相对说明相融合。为支持零样本组合图像检索研究，我们引入名为"上下文中常见物体的组合图像检索"的开放域基准数据集，这是首个包含每个查询对应多个真实值的组合图像检索数据集。实验表明，SEARLE在组合图像检索任务的两个主要数据集FashionIQ和CIRR以及提出的CIRCO上均展现出优于基线的性能。数据集、代码和模型已公开于https://github.com/miccunifi/SEARLE。