LLMs have demonstrated great capabilities in various NLP tasks. Different entities can further improve the performance of those LLMs on their specific downstream tasks by fine-tuning LLMs. When several entities have similar interested tasks, but their data cannot be shared because of privacy concerns regulations, federated learning (FL) is a mainstream solution to leverage the data of different entities. However, fine-tuning LLMs in federated learning settings still lacks adequate support from existing FL frameworks because it has to deal with optimizing the consumption of significant communication and computational resources, data preparation for different tasks, and distinct information protection demands. This paper first discusses these challenges of federated fine-tuning LLMs, and introduces our package FS-LLM as a main contribution, which consists of the following components: (1) we build an end-to-end benchmarking pipeline, automizing the processes of dataset preprocessing, federated fine-tuning execution, and performance evaluation on federated LLM fine-tuning; (2) we provide comprehensive federated parameter-efficient fine-tuning algorithm implementations and versatile programming interfaces for future extension in FL scenarios with low communication and computation costs, even without accessing the full model; (3) we adopt several accelerating and resource-efficient operators for fine-tuning LLMs with limited resources and the flexible pluggable sub-routines for interdisciplinary study. We conduct extensive experiments to validate the effectiveness of FS-LLM and benchmark advanced LLMs with state-of-the-art parameter-efficient fine-tuning algorithms in FL settings, which also yields valuable insights into federated fine-tuning LLMs for the research community. To facilitate further research and adoption, we release FS-LLM at https://github.com/alibaba/FederatedScope/tree/llm.

翻译：大语言模型（LLMs）已在各类自然语言处理任务中展现出强大能力。不同实体可通过微调LLMs进一步提升其在特定下游任务上的性能。当多个实体面临相似任务但数据因隐私法规无法共享时，联邦学习（FL）成为利用多方数据的核心解决方案。然而，现有FL框架对LLMs的联邦微调支持不足，因其需解决通信与计算资源消耗优化、多任务数据准备及差异化信息保护需求等挑战。本文首先探讨联邦微调LLMs的上述挑战，并提出核心贡献——FS-LLM工具包，其包含以下组件：(1) 构建端到端基准测试流水线，自动化实现数据集预处理、联邦微调执行及联邦LLM微调性能评估；(2) 提供全面的联邦参数高效微调算法实现与通用编程接口，支持低通信与计算开销场景下的未来扩展，甚至可在无需访问完整模型时运行；(3) 采用多种加速与资源高效算子，在有限资源下实现LLM微调，并设计可灵活插拔的跨学科研究子程序。通过大量实验验证FS-LLM的有效性，并在FL场景下基于最先进的参数高效微调算法对先进LLMs进行基准测试，为联邦微调LLMs领域贡献了重要洞见。为促进后续研究与应用，我们已在https://github.com/alibaba/FederatedScope/tree/llm 开源FS-LLM。