Catastrophic Forgetting (CF) is a prominent issue in continual learning. Parameter isolation addresses this challenge by masking a sub-network for each task to mitigate interference with old tasks. However, these sub-networks are constructed relying on weight magnitude, which does not necessarily correspond to the importance of weights, resulting in maintaining unimportant weights and constructing redundant sub-networks. To overcome this limitation, inspired by information bottleneck, which removes redundancy between adjacent network layers, we propose \textbf{\underline{I}nformation \underline{B}ottleneck \underline{M}asked sub-network (IBM)} to eliminate redundancy within sub-networks. Specifically, IBM accumulates valuable information into essential weights to construct redundancy-free sub-networks, not only effectively mitigating CF by freezing the sub-networks but also facilitating new tasks training through the transfer of valuable knowledge. Additionally, IBM decomposes hidden representations to automate the construction process and make it flexible. Extensive experiments demonstrate that IBM consistently outperforms state-of-the-art methods. Notably, IBM surpasses the state-of-the-art parameter isolation method with a 70\% reduction in the number of parameters within sub-networks and an 80\% decrease in training time.

翻译：灾难性遗忘是持续学习中的一个显著问题。参数隔离方法通过为每个任务掩码出一个子网络来缓解对旧任务的干扰，然而，这些子网络基于权重幅值构建，而权重幅值并不必然对应权重的重要性，导致保留了不重要的权重并构建了冗余子网络。为克服这一局限，受信息瓶颈（通过消除相邻网络层间的冗余）的启发，我们提出**信息瓶颈掩码子网络（IBM）** 以消除子网络内部的冗余。具体而言，IBM将有价值的信息积累到关键权重中，从而构建无冗余的子网络，这不仅通过冻结子网络有效缓解了灾难性遗忘，还通过传递有价值的知识促进新任务训练。此外，IBM通过分解隐层表示来自动化构建过程并提升其灵活性。大量实验表明，IBM始终优于现有最先进方法。值得注意的是，IBM在子网络参数数量减少70%、训练时间减少80%的情况下，超越了最先进的参数隔离方法。