The surge in interest and application of large language models (LLMs) has sparked a drive to fine-tune these models to suit specific applications, such as finance and medical science. However, concerns regarding data privacy have emerged, especially when multiple stakeholders aim to collaboratively enhance LLMs using sensitive data. In this scenario, federated learning becomes a natural choice, allowing decentralized fine-tuning without exposing raw data to central servers. Motivated by this, we investigate how data privacy can be ensured in LLM fine-tuning through practical federated learning approaches, enabling secure contributions from multiple parties to enhance LLMs. Yet, challenges arise: 1) despite avoiding raw data exposure, there is a risk of inferring sensitive information from model outputs, and 2) federated learning for LLMs incurs notable communication overhead. To address these challenges, this article introduces DP-LoRA, a novel federated learning algorithm tailored for LLMs. DP-LoRA preserves data privacy by employing a Gaussian mechanism that adds noise in weight updates, maintaining individual data privacy while facilitating collaborative model training. Moreover, DP-LoRA optimizes communication efficiency via low-rank adaptation, minimizing the transmission of updated weights during distributed training. The experimental results across medical, financial, and general datasets using various LLMs demonstrate that DP-LoRA effectively ensures strict privacy constraints while minimizing communication overhead.

翻译：大型语言模型（LLM）的兴趣与应用激增，推动了针对特定领域（如金融与医学）的模型微调需求。然而，数据隐私问题随之凸显，尤其当多方希望利用敏感数据协同改进LLM时。在此场景下，联邦学习成为自然选择，它允许在无需向中心服务器暴露原始数据的前提下进行去中心化微调。受此驱动，本研究探索如何通过实用的联邦学习方法在LLM微调中确保数据隐私，实现多方安全协作增强LLM。但挑战依然存在：1）尽管避免了原始数据暴露，仍存在从模型输出推断敏感信息的风险；2）LLM的联邦学习会产生显著的通信开销。为应对这些挑战，本文提出DP-LoRA——一种专为LLM设计的新型联邦学习算法。DP-LoRA通过高斯机制在权重更新中注入噪声来保护数据隐私，在促进协同模型训练的同时保障个体数据隐私。此外，DP-LoRA通过低秩适配优化通信效率，最大限度减少分布式训练中的权重更新传输量。在医学、金融及通用数据集上使用多种LLM的实验结果表明，DP-LoRA能在满足严格隐私约束的同时有效降低通信开销。