Continual learning (CL) for large language models (LLMs) aims to enable sequential knowledge acquisition without catastrophic forgetting. Memory replay methods are widely used for their practicality and effectiveness, but most rely on fixed, step-based heuristics that often misalign with the model's actual learning progress, since identical training steps can result in varying degrees of parameter change. Motivated by recent findings that LLM forgetting mirrors the Ebbinghaus human forgetting curve, we propose FOREVER (FORgEtting curVe-inspired mEmory Replay), a novel CL framework that aligns replay schedules with a model-centric notion of time. FOREVER defines model time using the magnitude of optimizer updates, allowing forgetting curve-inspired replay intervals to align with the model's internal evolution rather than raw training steps. Building on this approach, FOREVER incorporates a forgetting curve-based replay scheduler to determine when to replay and an intensity-aware regularization mechanism to adaptively control how to replay. Extensive experiments on three CL benchmarks and models ranging from 0.6B to 13B parameters demonstrate that FOREVER consistently mitigates catastrophic forgetting.

翻译：语言模型持续学习旨在实现顺序知识获取而不发生灾难性遗忘。记忆回放方法因其实用性和有效性被广泛采用，但多数依赖固定的基于训练步数的启发式策略，这些策略常与模型实际学习进度失准，因为相同的训练步数可能导致不同程度的参数变化。受近期研究发现语言模型遗忘遵循艾宾浩斯人类遗忘曲线的启发，我们提出FOREVER（基于遗忘曲线的记忆回放），这是一种新颖的持续学习框架，其回放调度与以模型为中心的时间概念对齐。FOREVER通过优化器更新幅度定义模型时间，使得基于遗忘曲线的回放间隔能与模型内部演化而非原始训练步数保持同步。基于此方法，FOREVER整合了基于遗忘曲线的回放调度器（决定何时回放）和强度感知正则化机制（自适应控制如何回放）。在三个持续学习基准测试和参数规模从0.6B到13B的模型上进行的大量实验表明，FOREVER能持续有效缓解灾难性遗忘。