The goal of Universal Cross-Domain Retrieval (UCDR) is to achieve robust performance in generalized test scenarios, wherein data may belong to strictly unknown domains and categories during training. Recently, pre-trained models with prompt tuning have shown strong generalization capabilities and attained noteworthy achievements in various downstream tasks, such as few-shot learning and video-text retrieval. However, applying them directly to UCDR may not sufficiently to handle both domain shift (i.e., adapting to unfamiliar domains) and semantic shift (i.e., transferring to unknown categories). To this end, we propose Prompting-to-Simulate (ProS), the first method to apply prompt tuning for UCDR. ProS employs a two-step process to simulate Content-aware Dynamic Prompts (CaDP) which can impact models to produce generalized features for UCDR. Concretely, in Prompt Units Learning stage, we introduce two Prompt Units to individually capture domain and semantic knowledge in a mask-and-align way. Then, in Context-aware Simulator Learning stage, we train a Content-aware Prompt Simulator under a simulated test scenarios to produce the corresponding CaDP. Extensive experiments conducted on three benchmark datasets show that our method achieves new state-of-the-art performance without bringing excessive parameters. Our method is publicly available at https://anonymous.4open.science/r/ProS
翻译:通用跨域检索(UCDR)的目标是在泛化测试场景中实现鲁棒性能,其中训练阶段的数据可能属于严格未知的域和类别。近年来,通过提示调优的预训练模型展现出强大的泛化能力,并在少样本学习、视频-文本检索等各类下游任务中取得显著成果。然而,将其直接应用于UCDR可能不足以同时处理域偏移(即适应未知域)和语义偏移(即迁移至未知类别)。为此,我们提出模拟提示(ProS),这是首个将提示调优应用于UCDR的方法。ProS通过两步流程生成内容感知动态提示(CaDP),这些提示能够影响模型产生适用于UCDR的泛化特征。具体而言,在提示单元学习阶段,我们引入两个提示单元,以掩码对齐方式分别捕获域知识和语义知识。随后在上下文感知模拟器学习阶段,我们在模拟测试场景下训练内容感知提示模拟器,以生成相应的CaDP。在三个基准数据集上进行的大量实验表明,我们的方法在未引入过多参数的情况下达到了新的最优性能。该方法已在 https://anonymous.4open.science/r/ProS 公开。