Despite the continuous research and evolution of language models, they sometimes underperform previous versions. Existing approaches to overcome these challenges are resource-intensive, highlighting the need for alternatives that enable immediate action. We assume that each language model has a local module inside that is suitable for a specific function. First, this work identifies a set of modules showing consistent and local activation changes under an inference workload through activation-based analysis. Subsequently, we transplant an internal module that is properly activated for a specific task into the target model, leading to immediate and measurable functional changes without additional training or fine-tuning. To experimentally demonstrate the effectiveness of the transplant technique, we quantify the relationship between transplant strength and performance improvement under different conditions for two language models. In the cross-generation setting, we find that transplanting activation-selected modules can substantially improve the underperforming model, reaching up to twice the target baseline and achieving gap-based recovery above 100%. Moreover, in transplant experiments between a base model and its instruction-tuned counterpart, transplantation improves the underperforming model toward the stronger baseline, yielding up to about 2.33 times the target baseline with gap-based recovery reaching up to 100% in the best case. These results show that meaningful capacity transfer can be realized through the implantation of highly localized modules implied by language models. Overall, this work provides empirical evidence for task-localized modularity in language models and presents a new research area: model transplantation.

翻译：尽管语言模型的研究持续演进，它们有时仍会逊色于先前版本。现有克服这些挑战的方法往往资源密集，这凸显了对能够实现即时行动的替代方案的需求。我们假设每个语言模型内部都存在一个适合特定功能的局部模块。首先，本研究通过基于激活的分析，识别出一组在推理工作负载下表现出稳定且局部激活变化的模块。随后，我们将针对特定任务被恰当激活的内部模块移植到目标模型中，从而在不进行额外训练或微调的情况下，实现即时且可测量的功能改变。为了通过实验验证移植技术的有效性，我们量化了两种语言模型在不同条件下移植强度与性能提升之间的关系。在跨代设置中，我们发现移植经激活选择的模块可以显著提升表现不佳的模型，达到目标基线性能的两倍，并以差距为基础的恢复率超过100%。此外，在基础模型与其指令微调对应模型之间的移植实验中，移植将表现不佳的模型向更强的基线提升，在最佳情况下达到目标基线约2.33倍，差距恢复率高达100%。这些结果表明，通过植入语言模型所隐含的高度局部化模块，可以实现有意义的能力迁移。总体而言，本研究为语言模型中任务局部化的模块性提供了实证证据，并提出了一个新的研究领域：模型移植。