Universal domain adaptation (UniDA) aims to address domain and category shifts across data sources. Recently, due to more stringent data restrictions, researchers have introduced source-free UniDA (SF-UniDA). SF-UniDA methods eliminate the need for direct access to source samples when performing adaptation to the target domain. However, existing SF-UniDA methods still require an extensive quantity of labeled source samples to train a source model, resulting in significant labeling costs. To tackle this issue, we present a novel plug-and-play classifier-oriented calibration (COCA) method. COCA, which exploits textual prototypes, is designed for the source models based on few-shot learning with vision-language models (VLMs). It endows the VLM-powered few-shot learners, which are built for closed-set classification, with the unknown-aware ability to distinguish common and unknown classes in the SF-UniDA scenario. Crucially, COCA is a new paradigm to tackle SF-UniDA challenges based on VLMs, which focuses on classifier instead of image encoder optimization. Experiments show that COCA outperforms state-of-the-art UniDA and SF-UniDA models.
翻译:通用域自适应旨在解决跨数据源的域偏移和类别偏移问题。近年来,由于更严格的数据限制,研究者引入了无源通用域自适应方法。该类方法在执行目标域自适应时无需直接访问源样本。然而,现有SF-UniDA方法仍需大量标注的源样本来训练源模型,从而导致显著的标注成本。为解决这一问题,我们提出一种新颖的即插即用型面向分类器校准方法COCA。该基于文本原型的方法专为采用视觉语言模型的少样本学习源模型设计,使基于VLMs构建的闭集分类少样本学习器具备未知感知能力,能够在SF-UniDA场景中区分常见类别与未知类别。关键在于,COCA是一种基于VLMs解决SF-UniDA挑战的新范式,其聚焦于分类器优化而非图像编码器优化。实验表明,COCA优于当前最先进的UniDA和SF-UniDA模型。