Most existing methods for multi-source unsupervised domain adaptation (UDA) rely on a common encoder to extract domain-invariant features. However, learning such an encoder involves updating the parameters of the entire network, which makes the optimization difficult and computationally expensive, particularly when coupled with min-max objectives. Inspired by recent advances in prompt learning that adapts high-capacity models for downstream tasks in a computationally economic way, we introduce Multi-Prompt Alignment (MPA), a simple yet efficient two-stage framework for multi-source UDA. Given a source and target domain pair, MPA first trains an individual prompt to minimize the domain gap through a contrastive loss. Then, MPA derives a low-dimensional latent space through an auto-encoding process that maximizes the agreement of multiple learned prompts. The resulting embeddings further facilitate generalization to unseen domains, making MPA suitable for test time adaptation. Extensive experiments show that our method achieves state-of-the-art results on popular datasets while requiring substantially fewer tunable parameters. Specifically on DomainNet, the most challenging UDA dataset, MPA achieves the highest reported average accuracy of 54.1% with only 15.9M parameters trained.
翻译:大多数现有的多源无监督域适应(UDA)方法依赖于一个共享编码器来提取域不变特征。然而,学习此类编码器涉及更新整个网络的参数,这使得优化困难且计算成本高昂,尤其是在结合最小-最大目标时。受近期提示学习进展的启发,提示学习能够以经济的方式使高容量模型适应下游任务,我们提出了多提示对齐(MPA),一种简单而高效的两阶段多源UDA框架。对于给定的源域和目标域对,MPA首先训练一个单独的提示,通过对比损失最小化域差距。然后,MPA通过一个自编码过程导出低维潜在空间,该过程最大化多个学习提示的一致性。得到的嵌入进一步促进对未见域泛化,使MPA适用于测试时适应。大量实验表明,我们的方法在流行数据集上达到了最先进的结果,同时所需的可调参数大幅减少。具体来说,在最具挑战性的UDA数据集DomainNet上,MPA仅训练15.9M个参数就实现了54.1%的最高报告平均准确率。