Catastrophic forgetting remains a critical challenge in the field of continual learning, where neural networks struggle to retain prior knowledge while assimilating new information. Most existing studies emphasize mitigating this issue only when encountering new tasks, overlooking the significance of the pre-task phase. Therefore, we shift the attention to the current task learning stage, presenting a novel framework, C&F (Create and Find Flatness), which builds a flat training space for each task in advance. Specifically, during the learning of the current task, our framework adaptively creates a flat region around the minimum in the loss landscape. Subsequently, it finds the parameters' importance to the current task based on their flatness degrees. When adapting the model to a new task, constraints are applied according to the flatness and a flat space is simultaneously prepared for the impending task. We theoretically demonstrate the consistency between the created and found flatness. In this manner, our framework not only accommodates ample parameter space for learning new tasks but also preserves the preceding knowledge of earlier tasks. Experimental results exhibit C&F's state-of-the-art performance as a standalone continual learning approach and its efficacy as a framework incorporating other methods. Our work is available at https://github.com/Eric8932/Create-and-Find-Flatness.

翻译：灾难性遗忘仍然是持续学习领域的关键挑战，神经网络在学习新信息时难以保留已有知识。现有研究大多仅关注遇到新任务时的缓解策略，却忽视了任务前阶段的重要性。为此，我们将注意力转向当前任务学习阶段，提出名为C&F（创建与寻找平坦性）的新型框架，该框架为每个任务预先构建平坦训练空间。具体而言，在学习当前任务时，我们的框架自适应地在损失景观的最小值周围创建平坦区域，随后根据参数的平坦程度确定其对当前任务的重要性。当模型适应新任务时，系统依据平坦性施加约束，并同时为即将到来的任务准备平坦空间。我们从理论上证明了所创建平坦性与所寻找平坦性的一致性。通过这种方式，本框架不仅为学习新任务保留了充足的参数空间，还能保留先前任务的知识。实验结果表明，C&F作为独立持续学习方法展现了最先进的性能，作为集成其他方法的框架亦表现优异。我们的工作已开源至https://github.com/Eric8932/Create-and-Find-Flatness。