We introduce Neural Organ Transplantation (NOT), a modular adaptation framework that enables trained transformer layers to function as reusable transferable checkpoints for domain adaptation. Unlike conventional fine-tuning approaches that tightly couple trained parameters to specific model instances and training data, NOT extracts contiguous layer subsets ("donor organs") from pre-trained models, trains them independently on domain-specific data, and saves them as standalone checkpoint files that can be transplanted into compatible recipient models without access to the original training data. Through experiments on three decoder-only transformer architectures spanning 124M to 20B parameters (GPT-2, TinyLlama, and GPT-OSS), we demonstrate that donor transplantation substantially outperforms existing adaptation methods, achieving an order-of-magnitude improvement in perplexity over LoRA while training significantly faster. The method exhibits position dependence, with early insertion positions yielding optimal results. Cross-domain transfer at billion-parameter scale reveals unexpected regularization benefits. These findings demonstrate that transformer middle layers can support efficient modular transfer for decoder-only architectures, enabling privacy-preserving expertise sharing through checkpoint distribution. We note that this approach is currently limited to decoder-only models; preliminary experiments on encoder-based architectures show reduced effectiveness.

翻译：本文提出神经器官移植（NOT）——一种模块化适配框架，使训练后的Transformer层能够作为可复用的可迁移检查点用于领域适配。与传统微调方法将训练参数紧密耦合至特定模型实例和训练数据不同，NOT从预训练模型中提取连续层子集（“供体器官”），在领域特定数据上独立训练后，将其保存为独立的检查点文件。这些检查点无需原始训练数据即可移植到兼容的接收模型中。通过对三种仅含解码器的Transformer架构（GPT-2、TinyLlama和GPT-OSS，参数量覆盖1.24亿至200亿）的实验证明，供体移植显著优于现有适配方法：在困惑度指标上较LoRA提升一个数量级，同时训练速度大幅加快。该方法呈现位置依赖性，早期插入位置能获得最优效果。在十亿参数规模上的跨领域迁移实验揭示了意外的正则化效益。这些发现表明Transformer中间层能够支持仅解码器架构的高效模块化迁移，通过检查点分发实现隐私保护的专业知识共享。需要说明的是，该方法目前仅适用于仅解码器模型；在基于编码器的架构上的初步实验显示其有效性有所降低。