Real-world applications require the classification model to adapt to new classes without forgetting old ones. Correspondingly, Class-Incremental Learning (CIL) aims to train a model with limited memory size to meet this requirement. Typical CIL methods tend to save representative exemplars from former classes to resist forgetting, while recent works find that storing models from history can substantially boost the performance. However, the stored models are not counted into the memory budget, which implicitly results in unfair comparisons. We find that when counting the model size into the total budget and comparing methods with aligned memory size, saving models do not consistently work, especially for the case with limited memory budgets. As a result, we need to holistically evaluate different CIL methods at different memory scales and simultaneously consider accuracy and memory size for measurement. On the other hand, we dive deeply into the construction of the memory buffer for memory efficiency. By analyzing the effect of different layers in the network, we find that shallow and deep layers have different characteristics in CIL. Motivated by this, we propose a simple yet effective baseline, denoted as MEMO for Memory-efficient Expandable MOdel. MEMO extends specialized layers based on the shared generalized representations, efficiently extracting diverse representations with modest cost and maintaining representative exemplars. Extensive experiments on benchmark datasets validate MEMO's competitive performance. Code is available at: https://github.com/wangkiw/ICLR23-MEMO

翻译：现实世界应用要求分类模式适应新班级,而不会忘记旧班级。相应而言,分类强化学习(CIL)旨在培训一个记忆量有限的模型,以满足这一要求。典型的CIL方法往往使前班的代表性外延者避免忘记,而最近的工作发现,历史的存储模型可以大大提升业绩。然而,存储的模型没有计入记忆预算,这间接导致不公平的比较。我们发现,在计算总预算的模型规模和比较与记忆量一致的方法时,储蓄模型并非始终如一,特别是对于记忆量有限的案例而言。因此,我们需要从整体上评价不同记忆量的CIL方法,同时考虑测量的准确性和记忆量。另一方面,我们深陷在构建记忆缓冲中可以大大提高记忆效率。我们通过分析网络中不同层的影响,发现浅层和深层在CILL23中具有不同特征。我们为此提出一个简单而有效的基线,在可记忆节能扩展的MOdel 预算的情况下,保存模型不连贯。因此,我们需要从整体上评价不同的CLILO/平流的MEMO方法,同时考虑不同方法,在不同的缩缩缩缩缩缩缩。