Federated learning (FL) is a promising paradigm to enable collaborative model training with decentralized data. However, the training process of Large Language Models (LLMs) generally incurs the update of significant parameters, which limits the applicability of FL techniques to tackle the LLMs in real scenarios. Prompt tuning can significantly reduce the number of parameters to update, but it either incurs performance degradation or low training efficiency. The straightforward utilization of prompt tuning in the FL often raises non-trivial communication costs and dramatically degrades performance. In addition, the decentralized data is generally non-Independent and Identically Distributed (non-IID), which brings client drift problems and thus poor performance. This paper proposes a Parameter-efficient prompt Tuning approach with Adaptive Optimization, i.e., FedPepTAO, to enable efficient and effective FL of LLMs. First, an efficient partial prompt tuning approach is proposed to improve performance and efficiency simultaneously. Second, a novel adaptive optimization method is developed to address the client drift problems on both the device and server sides to enhance performance further. Extensive experiments based on 10 datasets demonstrate the superb performance (up to 60.8\% in terms of accuracy) and efficiency (up to 97.59\% in terms of training time) of FedPepTAO compared with 9 baseline approaches. Our code is available at https://github.com/llm-eff/FedPepTAO.

翻译：联邦学习（FL）是一种利用分布式数据实现协同模型训练的有效范式。然而，大语言模型（LLMs）的训练过程通常需要更新大量参数，这限制了FL技术在真实场景中处理LLMs的适用性。提示调优能显著减少需更新的参数量，但会引发性能下降或训练效率低下的问题。在FL中直接使用提示调优往往会导致非平凡的通信成本并严重降低性能。此外，分布式数据通常呈非独立同分布（non-IID），这带来了客户端漂移问题进而导致性能劣化。本文提出一种结合自适应优化的参数高效提示调优方法（FedPepTAO），以实现LLMs的高效联邦学习。首先，提出一种高效的部分提示调优方法以同时提升性能与效率；其次，开发了一种新型自适应优化方法来解决设备端与服务端的客户端漂移问题，从而进一步增强性能。基于10个数据集的广泛实验表明，与9种基线方法相比，FedPepTAO在准确率上最高提升60.8%，在训练时间上最高节省97.59%。我们的代码已开源至https://github.com/llm-eff/FedPepTAO。