Class-incremental learning is becoming more popular as it helps models widen their applicability while not forgetting what they already know. A trend in this area is to use a mixture-of-expert technique, where different models work together to solve the task. However, the experts are usually trained all at once using whole task data, which makes them all prone to forgetting and increasing computational burden. To address this limitation, we introduce a novel approach named SEED. SEED selects only one, the most optimal expert for a considered task, and uses data from this task to fine-tune only this expert. For this purpose, each expert represents each class with a Gaussian distribution, and the optimal expert is selected based on the similarity of those distributions. Consequently, SEED increases diversity and heterogeneity within the experts while maintaining the high stability of this ensemble method. The extensive experiments demonstrate that SEED achieves state-of-the-art performance in exemplar-free settings across various scenarios, showing the potential of expert diversification through data in continual learning.

翻译：类增量学习正日益流行，因为它能帮助模型扩展其应用范围，同时不遗忘已有知识。该领域的一个趋势是采用混合专家技术，即不同模型协同解决任务。然而，现有方法通常使用完整任务数据同时训练所有专家，导致每个专家都容易发生遗忘并增加计算负担。为克服这一局限，我们提出了一种名为SEED的新方法。SEED仅为每个任务选择单一最优专家，并仅使用该任务数据对该专家进行微调。为此，每个专家以高斯分布表示每个类别，并根据这些分布的相似性选择最优专家。由此，SEED在保持集成方法高稳定性的同时，增强了专家内部的多样性与异质性。大量实验表明，SEED在无样本存储设置下，于多种场景中均达到最优性能，揭示了通过数据实现专家多样化在持续学习中的潜力。