Active learning (AL) aims to improve model performance within a fixed labeling budget by choosing the most informative data points to label. Existing AL focuses on the single-domain setting, where all data come from the same domain (e.g., the same dataset). However, many real-world tasks often involve multiple domains. For example, in visual recognition, it is often desirable to train an image classifier that works across different environments (e.g., different backgrounds), where images from each environment constitute one domain. Such a multi-domain AL setting is challenging for prior methods because they (1) ignore the similarity among different domains when assigning labeling budget and (2) fail to handle distribution shift of data across different domains. In this paper, we propose the first general method, dubbed composite active learning (CAL), for multi-domain AL. Our approach explicitly considers the domain-level and instance-level information in the problem; CAL first assigns domain-level budgets according to domain-level importance, which is estimated by optimizing an upper error bound that we develop; with the domain-level budgets, CAL then leverages a certain instance-level query strategy to select samples to label from each domain. Our theoretical analysis shows that our method achieves a better error bound compared to current AL methods. Our empirical results demonstrate that our approach significantly outperforms the state-of-the-art AL methods on both synthetic and real-world multi-domain datasets. Code is available at https://github.com/Wang-ML-Lab/multi-domain-active-learning.

翻译：主动学习（AL）旨在通过选择最具信息量的数据点进行标注，在固定标注预算内提升模型性能。现有主动学习主要集中于单一领域场景，即所有数据来自同一领域（如同一个数据集）。然而，许多实际任务常涉及多个领域。例如，在视觉识别中，通常需要训练一个能跨不同环境（如不同背景）工作的图像分类器，其中每个环境的图像构成一个领域。这种多领域主动学习场景对现有方法构成挑战，因为它们（1）在分配标注预算时忽视了不同领域间的相似性，（2）未能处理跨领域数据的分布偏移。本文提出首个通用方法——复合主动学习（CAL），用于解决多领域主动学习问题。该方法明确考虑了问题中的领域级和实例级信息：CAL首先根据领域级重要性分配领域级预算，其重要性通过优化我们推导出的误差上界来估计；在获得领域级预算后，CAL再利用某种实例级查询策略从各领域选择样本进行标注。理论分析表明，与现有主动学习方法相比，我们的方法实现了更优的误差界。实验结果证明，在合成和真实世界的多领域数据集上，我们的方法显著优于最先进的主动学习方法。代码见 https://github.com/Wang-ML-Lab/multi-domain-active-learning。