A primary goal of class-incremental learning is to strike a balance between stability and plasticity, where models should be both stable enough to retain knowledge learned from previously seen classes, and plastic enough to learn concepts from new classes. While previous works demonstrate strong performance on class-incremental benchmarks, it is not clear whether their success comes from the models being stable, plastic, or a mixture of both. This paper aims to shed light on how effectively recent class-incremental learning algorithms address the stability-plasticity trade-off. We establish analytical tools that measure the stability and plasticity of feature representations, and employ such tools to investigate models trained with various algorithms on large-scale class-incremental benchmarks. Surprisingly, we find that the majority of class-incremental learning algorithms heavily favor stability over plasticity, to the extent that the feature extractor of a model trained on the initial set of classes is no less effective than that of the final incremental model. Our observations not only inspire two simple algorithms that highlight the importance of feature representation analysis, but also suggest that class-incremental learning approaches, in general, should strive for better feature representation learning.

翻译：类别增量学习的一个核心目标是平衡稳定性与可塑性：模型既需足够稳定以保留先前学习类别的知识，又需足够可塑以学习新类别的概念。尽管现有方法在类别增量基准测试中表现优异，但其成功究竟源于模型的稳定性、可塑性，还是两者共同作用，目前尚不明确。本文旨在揭示近期类别增量学习算法如何有效处理稳定性-可塑性权衡。我们建立了度量特征表征稳定性与可塑性的分析工具，并利用这些工具研究在大规模类别增量基准上经过多种算法训练的模型。令人惊讶的是，我们发现大多数类别增量学习算法严重偏向稳定性，以至于仅在初始类别集上训练的模型的特征提取器，其有效性不亚于最终增量模型。我们的观察结果不仅启发了两种强调特征表征分析重要性的简单算法，还表明类别增量学习方法总体上应致力于更优的特征表征学习。