Different from Composed Image Retrieval task that requires expensive labels for training task-specific models, Zero-Shot Composed Image Retrieval (ZS-CIR) involves diverse tasks with a broad range of visual content manipulation intent that could be related to domain, scene, object, and attribute. The key challenge for ZS-CIR tasks is to learn a more accurate image representation that has adaptive attention to the reference image for various manipulation descriptions. In this paper, we propose a novel context-dependent mapping network, named Context-I2W, for adaptively converting description-relevant Image information into a pseudo-word token composed of the description for accurate ZS-CIR. Specifically, an Intent View Selector first dynamically learns a rotation rule to map the identical image to a task-specific manipulation view. Then a Visual Target Extractor further captures local information covering the main targets in ZS-CIR tasks under the guidance of multiple learnable queries. The two complementary modules work together to map an image to a context-dependent pseudo-word token without extra supervision. Our model shows strong generalization ability on four ZS-CIR tasks, including domain conversion, object composition, object manipulation, and attribute manipulation. It obtains consistent and significant performance boosts ranging from 1.88% to 3.60% over the best methods and achieves new state-of-the-art results on ZS-CIR. Our code is available at https://github.com/Pter61/context-i2w.

翻译：摘要：与需要昂贵标签训练任务特定模型的组合图像检索任务不同，零样本组合图像检索（ZS-CIR）涉及多种任务，涵盖与领域、场景、对象和属性相关的广泛视觉内容操作意图。ZS-CIR任务的关键挑战在于学习更精准的图像表示，使其能够针对不同操作描述对参考图像产生自适应注意力。本文提出一种新颖的上下文相关映射网络Context-I2W，用于自适应地将与描述相关的图像信息转换为由描述构成的伪词标记，以实现精准的ZS-CIR。具体而言，意图视角选择器首先动态学习一种旋转规则，将相同图像映射到任务特定的操作视角；随后，视觉目标提取器在多个可学习查询的引导下，进一步捕获涵盖ZS-CIR任务主要目标的局部信息。这两个互补模块协同工作，在无需额外监督的情况下将图像映射为上下文相关的伪词标记。我们的模型在四项ZS-CIR任务（包括领域转换、对象组合、对象操作和属性操作）中展现出强大的泛化能力，相较于最佳方法获得1.88%至3.60%的持续显著性能提升，并在ZS-CIR上实现了新的最优结果。我们的代码开源在https://github.com/Pter61/context-i2w。