As an efficient alternative to conventional full finetuning, parameter-efficient finetuning (PEFT) is becoming the prevailing method to adapt pretrained language models. In PEFT, a lightweight module is learned on each dataset while the underlying pretrained language model remains unchanged, resulting in multiple compact modules representing diverse skills when applied to various domains and tasks. In this paper, we propose to compose these parameter-efficient modules through linear arithmetic operations in the weight space, thereby integrating different module capabilities. Specifically, we first define addition and negation operators for the module, and then further compose these two basic operators to perform flexible arithmetic. Our approach requires \emph{no additional training} and enables highly flexible module composition. We apply different arithmetic operations to compose the parameter-efficient modules for (1) distribution generalization, (2) multi-tasking, (3) unlearning, and (4) domain transfer. Additionally, we extend our approach to detoxify Alpaca-LoRA, the latest instruction-tuned large language model based on LLaMA. Empirical results demonstrate that our approach produces new and effective parameter-efficient modules that significantly outperform existing ones across all settings.

翻译：作为传统全参数微调的高效替代方案，参数高效微调（PEFT）正成为适配预训练语言模型的主流方法。PEFT在每个数据集上学习轻量级模块，同时保持底层预训练语言模型不变，从而在应用于不同领域和任务时产生代表多样化技能的多个紧凑模块。本文提出通过权重空间中的线性算术运算组合这些参数高效模块，进而整合不同模块的能力。具体而言，我们首先为模块定义加法与取反运算符，随后进一步组合这两个基本运算符以执行灵活运算。该方法无需额外训练，且支持高度灵活的模块组合。我们应用不同算术运算组合参数高效模块以实现：(1)分布泛化、(2)多任务处理、(3)知识遗忘及(4)领域迁移。此外，我们将该方法扩展至对Alpaca-LoRA（基于LLaMA的最新指令微调大语言模型）进行去毒化处理。实验结果表明，我们的方法能生成新颖且有效的参数高效模块，在全部设置中均显著优于现有方法。