While biological intelligence grows organically as new knowledge is gathered throughout life, Artificial Neural Networks forget catastrophically whenever they face a changing training data distribution. Rehearsal-based Continual Learning (CL) approaches have been established as a versatile and reliable solution to overcome this limitation; however, sudden input disruptions and memory constraints are known to alter the consistency of their predictions. We study this phenomenon by investigating the geometric characteristics of the learner's latent space and find that replayed data points of different classes increasingly mix up, interfering with classification. Hence, we propose a geometric regularizer that enforces weak requirements on the Laplacian spectrum of the latent space, promoting a partitioning behavior. We show that our proposal, called Continual Spectral Regularizer (CaSpeR), can be easily combined with any rehearsal-based CL approach and improves the performance of SOTA methods on standard benchmarks. Finally, we conduct additional analysis to provide insights into CaSpeR's effects and applicability.

翻译：尽管生物智能在生命过程中随着新知识的积累而有机增长，人工神经网络在面对不断变化的训练数据分布时仍会出现灾难性遗忘。基于重演的持续学习方法已被证实是克服这一局限的通用可靠方案，但突发的输入干扰与记忆约束会破坏其预测一致性。我们通过研究学习者潜在空间的几何特征来探讨这一现象，发现不同类别的重演数据点会逐渐混杂，干扰分类过程。为此，我们提出一种几何正则化方法，对潜在空间的拉普拉斯谱施加弱约束，从而促进分区行为。实验表明，我们的方法——即持续谱正则化（CaSpeR）——可轻松与任何基于重演的持续学习方法结合，并提升现有最优方法在标准基准测试中的性能。最后，我们通过额外分析揭示了CaSpeR的作用机制与适用性。