Despite significant advances, the performance of state-of-the-art continual learning approaches hinges on the unrealistic scenario of fully labeled data. In this paper, we tackle this challenge and propose an approach for continual semi-supervised learning--a setting where not all the data samples are labeled. A primary issue in this scenario is the model forgetting representations of unlabeled data and overfitting the labeled samples. We leverage the power of nearest-neighbor classifiers to nonlinearly partition the feature space and flexibly model the underlying data distribution thanks to its non-parametric nature. This enables the model to learn a strong representation for the current task, and distill relevant information from previous tasks. We perform a thorough experimental evaluation and show that our method outperforms all the existing approaches by large margins, setting a solid state of the art on the continual semi-supervised learning paradigm. For example, on CIFAR-100 we surpass several others even when using at least 30 times less supervision (0.8% vs. 25% of annotations). Finally, our method works well on both low and high resolution images and scales seamlessly to more complex datasets such as ImageNet-100. The code is publicly available on https://github.com/kangzhiq/NNCSL

翻译：尽管取得了显著进展，现有持续学习方法的表现仍依赖于全标注数据这一不切实际的场景。本文针对这一挑战，提出了一种面向持续半监督学习（即并非所有数据样本均有标注）的方法。该场景的核心问题在于模型会遗忘未标注数据的表征，并对已标注样本产生过拟合。我们利用最近邻分类器的能力对特征空间进行非线性划分，并凭借其非参数特性灵活建模底层数据分布。这使得模型能够为当前任务学习强大的表征，并从历史任务中提取相关信息。通过全面的实验评估，我们证明该方法以较大优势超越了所有现有方法，为持续半监督学习范式树立了坚实的新标杆。例如，在CIFAR-100数据集上，即使所用监督信号减少至少30倍（标注量仅为0.8% vs. 25%），我们的方法仍优于其他多种方案。此外，该方法在低分辨率与高分辨率图像上均表现优异，并能无缝扩展至ImageNet-100等更复杂的数据集。代码已开源发布于 https://github.com/kangzhiq/NNCSL