Foundation models (e.g., CLIP or DINOv2) have shown their impressive learning and transfer capabilities in a wide range of visual tasks, by training on a large corpus of data and adapting to specific downstream tasks. It is, however, interesting that foundation models have not been fully explored for universal domain adaptation (UniDA), which is to learn models using labeled data in a source domain and unlabeled data in a target one, such that the learned models can successfully adapt to the target data. In this paper, we make comprehensive empirical studies of state-of-the-art UniDA methods using foundation models. We first observe that, unlike fine-tuning from ImageNet pre-trained models, as previous methods do, fine-tuning from foundation models yields significantly poorer results, sometimes even worse than training from scratch. While freezing the backbones, we demonstrate that although the foundation models greatly improve the performance of the baseline method that trains the models on the source data alone, existing UniDA methods generally fail to improve over the baseline. This suggests that new research efforts are very necessary for UniDA using foundation models. Based on these findings, we introduce \textit{CLIP distillation}, a parameter-free method specifically designed to distill target knowledge from CLIP models. The core of our \textit{CLIP distillation} lies in a self-calibration technique for automatic temperature scaling, a feature that significantly enhances the baseline's out-class detection capability. Although simple, our method outperforms previous approaches in most benchmark tasks, excelling in evaluation metrics including H-score/H$^3$-score and the newly proposed universal classification rate (UCR) metric. We hope that our investigation and the proposed simple framework can serve as a strong baseline to facilitate future studies in this field.

翻译：基础模型（如CLIP或DINOv2）通过在大规模数据上训练并适配特定下游任务，已在多种视觉任务中展现出卓越的学习与迁移能力。然而有趣的是，这些模型在通用域自适应（UniDA）领域尚未得到充分探索——该领域旨在利用源域中的标注数据与目标域中的未标注数据训练模型，使其能够成功适配目标数据。本文对基于基础模型的最先进UniDA方法进行了全面实证研究。我们首先发现：不同于以往方法从ImageNet预训练模型进行微调，从基础模型微调的结果显著更差，有时甚至不如从零开始训练。在冻结主干网络的情况下，虽然基础模型显著提升了仅用源数据训练的基线方法的性能，但现有UniDA方法通常无法超越该基线。这表明基于基础模型的UniDA亟需新的研究探索。基于此发现，我们提出了一种专为从CLIP模型中蒸馏目标知识而设计的无参数方法——CLIP蒸馏。其核心在于一种用于自动温度缩放的自校准技术，该技术显著增强了基线方法的离群类检测能力。尽管方法简单，但它在多数基准任务中优于先前方法，在H分数/H³分数及新提出的通用分类率（UCR）等评估指标上表现卓越。我们希望本项研究及所提出的简洁框架能作为强有力的基线，推动该领域的未来发展。