Real-world applications require the classification model to adapt to new classes without forgetting old ones. Correspondingly, Class-Incremental Learning (CIL) aims to train a model with limited memory size to meet this requirement. Typical CIL methods tend to save representative exemplars from former classes to resist forgetting, while recent works find that storing models from history can substantially boost the performance. However, the stored models are not counted into the memory budget, which implicitly results in unfair comparisons. We find that when counting the model size into the total budget and comparing methods with aligned memory size, saving models do not consistently work, especially for the case with limited memory budgets. As a result, we need to holistically evaluate different CIL methods at different memory scales and simultaneously consider accuracy and memory size for measurement. On the other hand, we dive deeply into the construction of the memory buffer for memory efficiency. By analyzing the effect of different layers in the network, we find that shallow and deep layers have different characteristics in CIL. Motivated by this, we propose a simple yet effective baseline, denoted as MEMO for Memory-efficient Expandable MOdel. MEMO extends specialized layers based on the shared generalized representations, efficiently extracting diverse representations with modest cost and maintaining representative exemplars. Extensive experiments on benchmark datasets validate MEMO's competitive performance. Code is available at: https://github.com/wangkiw/ICLR23-MEMO

翻译：现实应用要求分类模型能够适应新类别而不遗忘旧知识。相应地，类增量学习（CIL）旨在利用有限的内存大小训练模型以满足这一需求。典型的CIL方法倾向于保存先前类别的代表性样例来抵抗遗忘，而近年研究发现存储历史模型能大幅提升性能。然而，存储的模型并未计入内存预算，这隐含地导致了不公平的比较。我们发现，当将模型大小计入总预算并在对齐内存大小的情况下比较方法时，存储模型并非始终有效，尤其在内存预算有限的情况下。因此，我们需要在不同内存尺度上整体评估不同CIL方法，并同时考虑准确率和内存大小进行度量。另一方面，我们深入研究了内存缓冲区的构建以提高内存效率。通过分析网络中不同层的影响，我们发现浅层和深层在CIL中具有不同特性。受此启发，我们提出了一种简单有效的基线方法，命名为MEMO（内存高效可扩展模型）。MEMO基于共享的泛化表示扩展专用层，以适度代价高效提取多样化表示并保留代表性样例。在基准数据集上的大量实验验证了MEMO的竞争性能。代码见：https://github.com/wangkiw/ICLR23-MEMO