In the field of continual learning, models are designed to learn tasks one after the other. While most research has centered on supervised continual learning, recent studies have highlighted the strengths of self-supervised continual representation learning. The improved transferability of representations built with self-supervised methods is often associated with the role played by the multi-layer perceptron projector. In this work, we depart from this observation and reexamine the role of supervision in continual representation learning. We reckon that additional information, such as human annotations, should not deteriorate the quality of representations. Our findings show that supervised models when enhanced with a multi-layer perceptron head, can outperform self-supervised models in continual representation learning.

翻译：在持续学习领域，模型被设计为逐一学习任务。尽管大多数研究集中于监督式持续学习，但近期研究突显了自监督持续表示学习的优势。自监督方法构建的表示具有更强的可迁移性，这通常归因于多层感知机投影器所起的作用。本研究基于这一观察，重新审视监督在持续表示学习中的作用。我们认为，额外信息（如人工标注）不应损害表示的质量。研究结果表明，当监督模型配备多层感知机头部时，其在持续表示学习中可超越自监督模型。