Continual learning (CL) has traditionally focused on minimizing exemplar memory, a constraint often misaligned with modern systems where GPU time, not storage, is the primary bottleneck. This paper challenges this paradigm by investigating a more realistic regime: one where memory is abundant enough to mitigate forgetting, but full retraining from scratch remains prohibitively expensive. In this practical "middle ground", we find that the core challenge shifts from stability to plasticity, as models become biased toward prior tasks and struggle to learn new ones. Conversely, improved stability allows simple replay baselines to outperform the state-of-the-art methods at a fraction of the GPU cost. To address this newly surfaced trade-off, we propose Weight Space Consolidation, a lightweight method that combines (1) rank-based parameter resets to restore plasticity with (2) weight averaging to enhance stability. Validated on both class-incremental learning with image classifiers and continual instruction tuning with large language models, our approach outperforms strong baselines while matching the low computational cost of replay, offering a scalable alternative to expensive full-retraining. These findings challenge long-standing CL assumptions and establish a new, cost-efficient baseline for real-world CL systems where exemplar memory is no longer the limiting factor.

翻译：传统持续学习（CL）研究主要聚焦于最小化样本存储需求，这一约束常与现代系统实际瓶颈——GPU计算时间而非存储容量——不相匹配。本文通过探索更现实的场景挑战这一范式：在内存资源足以缓解遗忘效应，但完全从头训练仍代价高昂的实用“中间地带”。我们发现，在此场景下核心挑战从稳定性问题转向可塑性问题——模型会过度偏向先前任务而难以学习新任务。反之，增强稳定性可使简单的回放基线方法以极低GPU成本超越现有先进方法。为解决这一新浮现的权衡问题，我们提出权重空间固化法：一种结合（1）基于秩的参数重置以恢复可塑性，与（2）权重平均以增强稳定性的轻量级方法。通过在图像分类器的类增量学习与大语言模型的持续指令调优任务上的验证，本方法在保持回放方法低计算成本优势的同时超越了强基线，为昂贵的全量重训练提供了可扩展的替代方案。这些发现挑战了长期存在的CL假设，并为样本存储不再受限的现实CL系统建立了新的成本效益基准。