Adapting pre-trained models to specialized tasks often leads to catastrophic forgetting, where new knowledge overwrites foundational capabilities. Existing methods either compromise performance on the new task or struggle to balance training stability with efficient reuse of pre-trained knowledge. We introduce a novel function-preserving expansion method that resolves this dilemma. Our technique expands model capacity by replicating pre-trained parameters within transformer submodules and applying a scaling correction that guarantees the expanded model is mathematically identical to the original at initialization, enabling stable training while exploiting existing knowledge. Empirically, our method eliminates the trade-off between plasticity and stability, matching the performance of full fine-tuning on downstream tasks without any degradation of the model's original capabilities. Furthermore, we demonstrate the modularity of our approach, showing that by selectively expanding a small subset of layers we can achieve the same performance as full fine-tuning at a fraction of the computational cost.

翻译：将预训练模型适配到特定任务时，常导致灾难性遗忘——新知识覆盖了基础能力。现有方法要么在新任务上性能受损，要么难以平衡训练稳定性与预训练知识的有效复用。我们提出一种新颖的函数保持式扩展方法，以解决这一困境。该技术通过复制Transformer子模块内的预训练参数并应用缩放校正来扩展模型容量，确保扩展后的模型在初始化时与原始模型数学等价，从而在利用已有知识的同时实现稳定训练。实验表明，我们的方法消除了可塑性与稳定性之间的权衡，在下游任务上达到了与全参数微调相当的性能，且未造成模型原始能力的任何退化。此外，我们验证了该方法的模块化特性：通过选择性扩展少量层，仅需部分计算成本即可实现与全参数微调相同的性能。