The current dominant paradigm when building a machine learning model is to iterate over a dataset over and over until convergence. Such an approach is non-incremental, as it assumes access to all images of all categories at once. However, for many applications, non-incremental learning is unrealistic. To that end, researchers study incremental learning, where a learner is required to adapt to an incoming stream of data with a varying distribution while preventing forgetting of past knowledge. Significant progress has been made, however, the vast majority of works focus on the fully supervised setting, making these algorithms label-hungry thus limiting their real-life deployment. To that end, in this paper, we make the first attempt to survey recently growing interest in label-efficient incremental learning. We identify three subdivisions, namely semi-, few-shot- and self-supervised learning to reduce labeling efforts. Finally, we identify novel directions that can further enhance label-efficiency and improve incremental learning scalability. Project website: https://github.com/kilickaya/label-efficient-il.

翻译：当前构建机器学习模型的主流范式是在数据集上反复迭代直至收敛。此类方法属于非增量学习，因其需要一次性获取所有类别的全部图像数据。然而在众多实际应用中，非增量学习并不现实。为此，研究者们提出了增量学习范式，要求学习器能适应数据分布不断变化的流式输入，同时防止遗忘历史知识。尽管该领域已取得显著进展，但绝大多数研究聚焦于全监督场景，导致这些算法对标注数据极度依赖，从而限制了实际应用部署。针对这一现状，本文首次系统综述了近年来备受关注的标签高效增量学习研究。我们识别出三个细分方向：半监督学习、小样本学习与自监督学习，旨在降低标注成本。最后，我们提出了若干能进一步提升标签效率与增量学习可扩展性的创新研究方向。项目网站：https://github.com/kilickaya/label-efficient-il。