We propose a learning problem involving adapting a pre-trained source model to the target domain for classifying all classes that appeared in the source data, using target data that covers only a partial label space. This problem is practical, as it is unrealistic for the target end-users to collect data for all classes prior to adaptation. However, it has received limited attention in the literature. To shed light on this issue, we construct benchmark datasets and conduct extensive experiments to uncover the inherent challenges. We found a dilemma -- on the one hand, adapting to the new target domain is important to claim better performance; on the other hand, we observe that preserving the classification accuracy of classes missing in the target adaptation data is highly challenging, let alone improving them. To tackle this, we identify two key directions: 1) disentangling domain gradients from classification gradients, and 2) preserving class relationships. We present several effective solutions that maintain the accuracy of the missing classes and enhance the overall performance, establishing solid baselines for holistic transfer of pre-trained models with partial target data.

翻译：我们提出一种学习问题：利用仅覆盖部分标签空间的目标数据，将预训练的源模型适配到目标域，以分类源数据中出现的所有类别。该问题具有实际意义，因为目标终端用户在适配前收集所有类别的数据并不现实。然而，这一问题在文献中得到的关注有限。为了揭示该问题，我们构建了基准数据集并开展大量实验，以探索其固有挑战。我们发现了一个两难困境：一方面，适配到新目标域对提升性能至关重要；另一方面，保留目标适配数据中缺失类别的分类准确率极具挑战性，更遑论提升它们。为解决此问题，我们确定了两个关键方向：1）将域梯度与分类梯度解耦，2）保留类别间关系。我们提出了若干有效方案，既能维持缺失类别的准确率，又能提升整体性能，为基于部分目标数据的预训练模型整体迁移建立了坚实的基线。