Low-Rank Adaptation (LoRA) is a popular technique for parameter-efficient fine-tuning of Large Language Models (LLMs). We study how different LoRA modules can be merged to achieve skill composition -- testing the performance of the merged model on a target task that involves combining multiple skills, each skill coming from a single LoRA. This setup is favorable when it is difficult to obtain training data for the target task and when it can be decomposed into multiple skills. First, we identify practically occurring use-cases that can be studied under the realm of skill composition, e.g. solving hard math-word problems with code, creating a bot to answer questions on proprietary manuals or about domain-specialized corpora. Our main contribution is to show that concatenation of LoRAs (CAT), which optimally weights LoRAs that were individually trained on different skills, outperforms existing model- and data- merging techniques; for instance on math-word problems, CAT beats these methods by an average of 43% and 12% respectively. Thus, this paper advocates model merging as an efficient way to solve compositional tasks and underscores CAT as a simple, compute-friendly and effective procedure. To our knowledge, this is the first work demonstrating the superiority of model merging over data mixing for binary skill composition tasks. Code and data are available at https://github.com/aksh555/LoRA-Soups

翻译：低秩自适应（LoRA）是一种用于大语言模型参数高效微调的流行技术。本研究探讨如何通过合并不同的LoRA模块实现技能组合——测试合并模型在目标任务上的性能，该任务需要组合多项技能，且每项技能均来自单个LoRA模块。当难以获取目标任务训练数据且该任务可分解为多项技能时，这种设置具有显著优势。首先，我们识别了实际应用中可纳入技能组合研究范畴的用例，例如：通过代码解决复杂数学文字问题、创建基于专有手册或专业领域语料库的问答机器人。我们的核心贡献在于证明：通过对针对不同技能独立训练的LoRA模块进行最优加权拼接（CAT），其性能优于现有的模型合并与数据混合技术；以数学文字问题为例，CAT方法平均分别超越这两种方法43%和12%。因此，本文主张将模型合并作为解决组合任务的有效途径，并强调CAT是一种简洁、计算友好且高效的解决方案。据我们所知，这是首个在二元技能组合任务中证明模型合并优于数据混合的研究。代码与数据已发布于https://github.com/aksh555/LoRA-Soups