Continual learning (CL) studies how models acquire tasks sequentially while retaining previously learned knowledge. Despite substantial progress in benchmarking CL methods, comparative evaluations typically keep the fine-tuning regime fixed. In this paper, we argue that the fine-tuning regime, defined by the trainable parameter subspace, is itself a key evaluation variable. We formalize adaptation regimes as projected optimization over fixed trainable subspaces, showing that changing the trainable depth alters the effective update signal through which both current task fitting and knowledge preservation operate. This analysis motivates the hypothesis that method comparisons need not be invariant across regimes. We test this hypothesis in task incremental CL, five trainable depth regimes, and four standard methods: online EWC, LwF, SI, and GEM. Across five benchmark datasets, namely MNIST, Fashion MNIST, KMNIST, QMNIST, and CIFAR-100, and across 11 task orders per dataset, we find that the relative ranking of methods is not consistently preserved across regimes. We further show that deeper adaptation regimes are associated with larger update magnitudes, higher forgetting, and a stronger relationship between the two. These results show that comparative conclusions in CL can depend strongly on the chosen fine-tuning regime, motivating regime-aware evaluation protocols that treat trainable depth as an explicit experimental factor.

翻译：持续学习（CL）研究模型如何顺序获取新任务，同时保留先前学到的知识。尽管在持续学习方法的基准测试方面取得了显著进展，但比较评估通常固定微调机制。本文提出，由可训练参数子空间定义的微调机制本身是一个关键的评估变量。我们将适应机制形式化为在固定可训练子空间上的投影优化，表明改变可训练深度会改变影响当前任务拟合与知识保留两者的有效更新信号。这一分析引出假设：方法比较在不同机制下不必保持一致。我们在任务增量持续学习、五种可训练深度机制以及四种标准方法（在线EWC、LwF、SI和GEM）中检验该假设。在五个基准数据集（MNIST、Fashion MNIST、KMNIST、QMNIST和CIFAR-100）上，每个数据集包含11种任务顺序，我们发现方法的相对排名在不同机制下并不一致。我们进一步表明，更深的适应机制与更大的更新幅度、更高的遗忘率以及两者之间更强的相关性相关联。这些结果表明，持续学习中的比较结论可能强烈依赖于所选的微调机制，这促使我们提出机制感知的评估协议，将可训练深度作为显式的实验因素。