Given an unlabeled dataset and an annotation budget, we study how to selectively label a fixed number of instances so that semi-supervised learning (SSL) on such a partially labeled dataset is most effective. We focus on selecting the right data to label, in addition to usual SSL's propagating labels from labeled data to the rest unlabeled data. This instance selection task is challenging, as without any labeled data we do not know what the objective of learning should be. Intuitively, no matter what the downstream task is, instances to be labeled must be representative and diverse: The former would facilitate label propagation to unlabeled data, whereas the latter would ensure coverage of the entire dataset. We capture this idea by selecting cluster prototypes, either in a pretrained feature space, or along with feature optimization, both without labels. Our unsupervised selective labeling consistently improves SSL methods over state-of-the-art active learning given labeled data, by 8 to 25 times in label efficiency. For example, it boosts FixMatch by 10% (14%) in accuracy on CIFAR-10 (ImageNet-1K) with 0.08% (0.2%) labeled data, demonstrating that small computation spent on selecting what data to label brings significant gain especially under a low annotation budget. Our work sets a new standard for practical and efficient SSL.

翻译：给定一个无标签数据集和一个标注预算，我们研究如何选择性标注固定数量的样本，使得在部分标注数据集上进行的半监督学习（SSL）最为有效。除了常规SSL中从标注数据向剩余无标注数据传播标签外，我们着重于选择正确的数据来标注。这一实例选择任务具有挑战性，因为缺乏任何标注数据时，我们无法确定学习目标。直观上，无论下游任务为何，待标注样本必须兼具代表性与多样性：前者有助于标签向无标注数据传播，后者则确保覆盖整个数据集。我们通过选择聚类原型来捕捉这一思想，这些原型可来自预训练特征空间，也可伴随特征优化过程，两者均无需标签。我们的无监督选择性标注方法在标签效率上比基于给定标注数据的最优主动学习方法持续提升8至25倍。例如，在仅使用0.08%（CIFAR-10）和0.2%（ImageNet-1K）标注数据的情况下，该方法将FixMatch的准确率分别提升10%和14%，证明在有限标注预算下，投入少量计算资源进行数据选择能带来显著收益。本工作为实用高效的半监督学习树立了新标杆。