Supervised Fine-Tuning (SFT) of large language models often suffers from task interference and catastrophic forgetting. Recent approaches alleviate this issue by isolating task-critical parameters during training. However, these methods represent a static solution to a dynamic problem, assuming that parameter importance remains fixed once identified. In this work, we empirically demonstrate that parameter importance exhibits temporal drift over the course of training. To address this, we propose Evolving Parameter Isolation (EPI), a fine-tuning framework that adapts isolation decisions based on online estimates of parameter importance. Instead of freezing a fixed subset of parameters, EPI periodically updates isolation masks using gradient-based signals, enabling the model to protect emerging task-critical parameters while releasing outdated ones to recover plasticity. Experiments on diverse multi-task benchmarks demonstrate that EPI consistently reduces interference and forgetting compared to static isolation and standard fine-tuning, while improving overall generalization. Our analysis highlights the necessity of synchronizing isolation mechanisms with the evolving dynamics of learning diverse abilities.

翻译：大语言模型的监督微调常面临任务干扰与灾难性遗忘问题。近期方法通过隔离训练中任务关键参数缓解该问题，但这些方法本质上是针对动态问题的静态解决方案——假定参数重要性在识别后恒定不变。本研究通过实验证明，参数重要性在训练过程中存在时间漂移现象。为此，我们提出演化参数隔离（EPI）框架，该微调方法基于参数重要性的在线估计动态调整隔离决策。与冻结固定参数子集不同，EPI利用梯度信号周期性更新隔离掩码，既保护新涌现的任务关键参数，又释放过时参数以恢复可塑性。在多任务基准上的实验表明，相比静态隔离方法与标准微调策略，EPI能持续减少任务干扰与遗忘，同时提升整体泛化能力。我们的分析揭示了将隔离机制与学习多元能力的演化动态同步的必要性。