The recent surge of pervasive devices that generate dynamic data streams has underscored the necessity for learning systems to adapt continually to data distributional shifts. To tackle this challenge, the research community has put forth a spectrum of methodologies, including the demanding pursuit of class-incremental learning without replay data. In this study, we present MIND, a parameter isolation method that aims to significantly enhance the performance of replay-free solutions and achieve state-of-the-art results on several widely studied datasets. Our approach introduces two main contributions: two alternative distillation procedures that significantly improve the efficiency of MIND increasing the accumulated knowledge of each sub-network, and the optimization of the BachNorm layers across tasks inside the sub-networks. Overall, MIND outperforms all the state-of-the-art methods for rehearsal-free Class-Incremental learning (with an increment in classification accuracy of approx. +6% on CIFAR-100/10 and +10% on TinyImageNet/10) reaching up to approx. +40% accuracy in Domain-Incremental scenarios. Moreover, we ablated each contribution to demonstrate its impact on performance improvement. Our results showcase the superior performance of MIND indicating its potential for addressing the challenges posed by Class-incremental and Domain-Incremental learning in resource-constrained environments.

翻译：随着生成动态数据流的泛在设备激增，学习系统亟需持续适应数据分布变化。为应对这一挑战，研究界提出了多种方法论，其中包括无需重放数据的类增量学习这一高难度追求。本研究提出参数隔离方法MIND，旨在显著提升无重放方案的性能，并在多个广泛研究的数据集上取得最优结果。我们的方法包含两项主要贡献：两种可显著提高MIND效率的替代蒸馏流程，通过增强各子网络累积知识实现性能提升；以及子网络内部跨任务的批归一化层优化。总体而言，MIND在无重放类增量学习场景中超越所有现有最优方法（CIFAR-100/10分类准确率提升约+6%，TinyImageNet/10提升约+10%），在域增量场景中准确率提升最高达约+40%。此外，我们通过消融实验验证了每项贡献对性能提升的影响。实验结果表明MIND具有优越性能，展现出其在资源受限环境下应对类增量与域增量学习挑战的潜力。