While training large language models (LLMs) from scratch can generate models with distinct functionalities and strengths, it comes at significant costs and may result in redundant capabilities. Alternatively, a cost-effective and compelling approach is to merge existing pre-trained LLMs into a more potent model. However, due to the varying architectures of these LLMs, directly blending their weights is impractical. In this paper, we introduce the notion of knowledge fusion for LLMs, aimed at combining the capabilities of existing LLMs and transferring them into a single LLM. By leveraging the generative distributions of source LLMs, we externalize their collective knowledge and unique strengths, thereby potentially elevating the capabilities of the target model beyond those of any individual source LLM. We validate our approach using three popular LLMs with different architectures--Llama-2, MPT, and OpenLLaMA--across various benchmarks and tasks. Our findings confirm that the fusion of LLMs can improve the performance of the target model across a range of capabilities such as reasoning, commonsense, and code generation. Our code, model weights, and data are public at \url{https://github.com/fanqiwan/FuseLLM}.

翻译：尽管从头训练大语言模型（LLMs）可以生成具有不同功能和优势的模型，但这一过程成本高昂，且可能导致能力冗余。另一种经济高效的替代方案是将现有的预训练LLMs合并为一个更强大的模型。然而，由于这些LLMs的架构各异，直接混合其权重并不现实。本文提出了针对LLMs的知识融合概念，旨在整合现有LLMs的能力，并将其迁移至单一LLM。通过利用源LLMs的生成分布，我们将其集体知识与独特优势外化，从而有可能使目标模型的能力超越任何单个源LLM。我们使用三种不同架构的流行LLM——Llama-2、MPT和OpenLLaMA——在多个基准和任务上验证了该方法。实验结果证实，LLMs的知识融合能够提升目标模型在推理、常识和代码生成等多种能力上的性能。我们的代码、模型权重和数据集已公开于\url{https://github.com/fanqiwan/FuseLLM}。