Class-Incremental Learning (CIL) aims to solve the neural networks' catastrophic forgetting problem, which refers to the fact that once the network updates on a new task, its performance on previously-learned tasks drops dramatically. Most successful CIL methods incrementally train a feature extractor with the aid of stored exemplars, or estimate the feature distribution with the stored prototypes. However, the stored exemplars would violate the data privacy concerns, while the stored prototypes might not reasonably be consistent with a proper feature distribution, hindering the exploration of real-world CIL applications. In this paper, we propose a method of \textit{e}mbedding distillation and \textit{Ta}sk-oriented \textit{g}eneration (\textit{eTag}) for CIL, which requires neither the exemplar nor the prototype. Instead, eTag achieves a data-free manner to train the neural networks incrementally. To prevent the feature extractor from forgetting, eTag distills the embeddings of the network's intermediate blocks. Additionally, eTag enables a generative network to produce suitable features, fitting the needs of the top incremental classifier. Experimental results confirmed that our proposed eTag considerably outperforms the state-of-the-art methods on CIFAR-100 and ImageNet-sub\footnote{Our code is available in the Supplementary Materials.

翻译：摘要：类增量学习旨在解决神经网络的灾难性遗忘问题，即网络在新任务上更新后，其在先前学习任务上的性能会显著下降。大多数成功的类增量学习方法通过存储的样本增量训练特征提取器，或利用存储的原型估计特征分布。然而，存储的样本可能引发数据隐私问题，而存储的原型可能无法合理保持与恰当特征分布的一致性，从而阻碍了类增量学习在实际应用中的探索。本文提出了一种用于类增量学习的嵌入蒸馏与任务导向生成方法（eTag），该方法既不依赖样本也不依赖原型。相反，eTag以无数据方式增量训练神经网络。为防止特征提取器遗忘，eTag蒸馏网络中间块的嵌入表示。此外，eTag使生成网络能够产生适合任务需求的合理特征，以满足增量分类器的需求。实验结果表明，我们提出的eTag在CIFAR-100和ImageNet-sub数据集上显著优于当前最先进方法（代码见补充材料）。