The surge in interest and application of large language models (LLMs) has sparked a drive to fine-tune these models to suit specific applications, such as finance and medical science. However, concerns regarding data privacy have emerged, especially when multiple stakeholders aim to collaboratively enhance LLMs using sensitive data. In this scenario, federated learning becomes a natural choice, allowing decentralized fine-tuning without exposing raw data to central servers. Motivated by this, we investigate how data privacy can be ensured in LLM fine-tuning through practical federated learning approaches, enabling secure contributions from multiple parties to enhance LLMs. Yet, challenges arise: 1) despite avoiding raw data exposure, there is a risk of inferring sensitive information from model outputs, and 2) federated learning for LLMs incurs notable communication overhead. To address these challenges, this article introduces DP-LoRA, a novel federated learning algorithm tailored for LLMs. DP-LoRA preserves data privacy by employing a Gaussian mechanism that adds noise in weight updates, maintaining individual data privacy while facilitating collaborative model training. Moreover, DP-LoRA optimizes communication efficiency via low-rank adaptation, minimizing the transmission of updated weights during distributed training. The experimental results across medical, financial, and general datasets using various LLMs demonstrate that DP-LoRA effectively ensures strict privacy constraints while minimizing communication overhead.

翻译：大语言模型（LLM）的广泛应用引发了针对特定场景（如金融和医学）进行模型微调的热潮。然而，当多方利益相关者希望利用敏感数据协作增强LLM时，数据隐私问题随之凸显。联邦学习作为自然解决方案应运而生，它允许在无需向中央服务器暴露原始数据的情况下进行分布式微调。受此启发，本研究探究如何通过实用的联邦学习方法在LLM微调中保障数据隐私，实现多方安全协作增强LLM。但挑战依然存在：1）尽管避免了原始数据暴露，模型输出仍存在推断敏感信息的风险；2）LLM的联邦学习会产生显著通信开销。为解决上述问题，本文提出DP-LoRA——面向LLM的新型联邦学习算法。该算法通过高斯机制向权重更新添加噪声来保护数据隐私，在保持个体数据隐私的同时实现协作模型训练。此外，DP-LoRA采用低秩优化技术提升通信效率，在分布式训练中大幅减少更新权重的传输量。基于多种LLM在医学、金融及通用数据集上的实验结果表明，DP-LoRA在满足严格隐私约束的同时，有效降低了通信开销。