Federated fine-tuning is critical for improving the performance of large language models (LLMs) in handling domain-specific tasks while keeping training data decentralized and private. However, prior work has shown that clients' private data can actually be recovered via gradient inversion attacks. Existing privacy preservation techniques against such attacks typically entail performance degradation and high costs, making them ill-suited for clients with heterogeneous data distributions and device capabilities. In this paper, we propose SHE-LoRA, which integrates selective homomorphic encryption (SHE) and low-rank adaptation (LoRA) to enable efficient and privacy-preserving federated tuning of LLMs in cross-device environments. Based on model parameter sensitivity assessment, heterogeneous clients adaptively negotiate and select a subset of model parameters for homomorphic encryption. To ensure accurate model aggregation, we design a column-aware secure aggregation method and customized reparameterization techniques to align the aggregation results with the heterogeneous device capabilities of clients. Extensive experiments demonstrate that SHE-LoRA maintains performance comparable to non-private baselines, achieves strong resistance to state-of-the-art attacks, and significantly reduces communication overhead by 99.71% and encryption time by 99.87%, compared to HE baselines.

翻译：联邦微调对于提升大语言模型（LLM）处理领域特定任务的性能至关重要，同时能够保持训练数据的去中心化与隐私性。然而，已有研究表明，客户端的私有数据实际上可能通过梯度反演攻击被恢复。针对此类攻击的现有隐私保护技术通常会导致性能下降与成本高昂，使其难以适应数据分布与设备能力异构的客户端环境。本文提出SHE-LoRA，该方法融合选择性同态加密（SHE）与低秩自适应（LoRA），以实现在跨设备环境中高效且保护隐私的LLM联邦调优。基于模型参数敏感性评估，异构客户端通过自适应协商，选择模型参数的一个子集进行同态加密。为确保精确的模型聚合，我们设计了一种列感知的安全聚合方法及定制化的重参数化技术，使聚合结果与客户端的异构设备能力相匹配。大量实验表明，SHE-LoRA在保持与非隐私基线相当性能的同时，能够有效抵御最先进的攻击，并且相较于同态加密基线，通信开销降低了99.71%，加密时间减少了99.87%。