Class incremental learning (CIL) trains a network on sequential tasks with separated categories in each task but suffers from catastrophic forgetting, where models quickly lose previously learned knowledge when acquiring new tasks. The generalized CIL (GCIL) aims to address the CIL problem in a more real-world scenario, where incoming data have mixed data categories and unknown sample size distribution. Existing attempts for the GCIL either have poor performance or invade data privacy by saving exemplars. In this paper, we propose a new exemplar-free GCIL technique named generalized analytic continual learning (GACL). The GACL adopts analytic learning (a gradient-free training technique) and delivers an analytical (i.e., closed-form) solution to the GCIL scenario. This solution is derived via decomposing the incoming data into exposed and unexposed classes, thereby attaining a weight-invariant property, a rare yet valuable property supporting an equivalence between incremental learning and its joint training. Such an equivalence is crucial in GCIL settings as data distributions among different tasks no longer pose challenges to adopting our GACL. Theoretically, this equivalence property is validated through matrix analysis tools. Empirically, we conduct extensive experiments where, compared with existing GCIL methods, our GACL exhibits a consistently leading performance across various datasets and GCIL settings. Source code is available at https://github.com/CHEN-YIZHU/GACL.

翻译：类别增量学习（CIL）在按任务划分类别的序列任务上训练网络，但会遭受灾难性遗忘——模型在学习新任务时迅速丢失先前习得的知识。广义CIL（GCIL）旨在解决更贴近现实场景的CIL问题，其中输入数据包含混合的数据类别且样本规模分布未知。现有GCIL方法要么性能较差，要么通过保存示例数据侵犯数据隐私。本文提出一种新的无需示例的GCIL技术，称为广义解析持续学习（GACL）。GACL采用解析学习（一种无梯度训练技术），为GCIL场景提供解析（即闭式）解。该解法通过将输入数据分解为已暴露和未暴露类别实现，从而获得权重不变性——这种稀缺而宝贵的特性支持增量学习与其联合训练的等价性。在GCIL设置中，这种等价性至关重要，因为不同任务间的数据分布不再对采用GACL构成挑战。理论上，该等价性通过矩阵分析工具得以验证。实证方面，我们进行了大量实验：与现有GCIL方法相比，GACL在多种数据集和GCIL设置中均表现出持续领先的性能。源代码发布于 https://github.com/CHEN-YIZHU/GACL。