The recent surge in pervasive devices generating dynamic data streams has underscored the necessity for learning systems to adapt to data distributional shifts continually. To tackle this challenge, the research community has put forth a spectrum of methodologies, including the demanding pursuit of class-incremental learning without replay data. In this study, we present MIND, a parameter isolation method that aims to significantly enhance the performance of replay-free solutions and achieve state-of-the-art results on several widely studied datasets. Our approach introduces two main contributions: two alternative distillation procedures that significantly improve the efficiency of MIND increasing the accumulated knowledge of each sub-network, and the optimization of the BachNorm layers across tasks inside the sub-networks. Overall, MIND outperforms all the state-of-the-art methods for rehearsal-free Class-Incremental learning (with an increment in classification accuracy of approx. +6% on CIFAR-100/10 and +10% on TinyImageNet/10) reaching up to approx. +40% accuracy in Domain-Incremental scenarios. Moreover, we ablated each contribution to demonstrate its impact on performance improvement. Our results showcase the superior performance of MIND indicating its potential for addressing the challenges posed by Class-incremental and Domain-Incremental learning in resource-constrained environments.

翻译：近期，随着产生动态数据流的普适设备激增，学习系统持续适应数据分布变化的需求日益凸显。为应对这一挑战，研究社区提出了一系列方法，包括对无需回放数据的类增量学习这一高难度方向的探索。在本研究中，我们提出MIND方法——一种参数隔离方法，旨在显著提升无回放方案的性能，并在多个广泛使用的数据集上达到最先进水平。我们的方法包含两项主要创新：其一，两种可显著提高MIND效率的替代蒸馏流程，能增强每个子网络的累积知识；其二，对子网络内各任务间的批归一化层进行优化。总体而言，MIND在无重放类增量学习场景中优于所有现有方法（在CIFAR-100/10上的分类精度提升约+6%，在TinyImageNet/10上提升约+10%），在域增量场景中精度提升高达约+40%。此外，我们对每项贡献进行消融实验，以证明其对性能提升的影响。实验结果展示了MIND的卓越性能，表明其在资源受限环境中应对类增量学习与域增量学习挑战的潜力。