Recent compositional zero-shot learning (CZSL) methods adapt pre-trained vision-language models (VLMs) by constructing trainable prompts only for composed state-object pairs. Relying on learning the joint representation of seen compositions, these methods ignore the explicit modeling of the state and object, thus limiting the exploitation of pre-trained knowledge and generalization to unseen compositions. With a particular focus on the universality of the solution, in this work, we propose a novel paradigm for CZSL models that establishes three identification branches (i.e., Multi-Path) to jointly model the state, object, and composition. The presented Troika is our implementation that aligns the branch-specific prompt representations with decomposed visual features. To calibrate the bias between semantically similar multi-modal representations, we further devise a Cross-Modal Traction module into Troika that shifts the prompt representation towards the current visual content. We conduct extensive experiments on three popular benchmarks, where our method significantly outperforms existing methods in both closed-world and open-world settings.

翻译：近期组合零样本学习方法通过仅针对复合状态-对象对构建可训练提示，来适配预训练的视觉-语言模型。这些方法依赖学习已见组合的联合表征，忽视了状态与对象的显式建模，从而限制了预训练知识的利用以及对未见组合的泛化能力。本文聚焦于解决方案的通用性，提出了一种新型组合零样本学习范式，该范式建立三个识别分支（即多路径）以联合建模状态、对象及其组合。所提出的Troika将分支特定的提示表征与分解后的视觉特征对齐。为校准语义相似的多模态表征间的偏差，我们进一步在Troika中设计了跨模态牵引模块，将提示表征向当前视觉内容偏移。我们在三个主流基准上进行了广泛实验，结果表明我们的方法在封闭世界与开放世界设定下均显著优于现有方法。