Federated learning (FL) is a popular privacy-preserving paradigm that enables distributed clients to collaboratively train models with a central server while keeping raw data locally. In practice, distinct model architectures, varying data distributions, and limited resources across local clients inevitably cause model performance degradation and a slowdown in convergence speed. However, existing FL methods can only solve some of the above heterogeneous challenges and have obvious performance limitations. Notably, a unified framework has not yet been explored to overcome these challenges. Accordingly, we propose FedHPL, a parameter-efficient unified $\textbf{Fed}$erated learning framework for $\textbf{H}$eterogeneous settings based on $\textbf{P}$rompt tuning and $\textbf{L}$ogit distillation. Specifically, we employ a local prompt tuning scheme that leverages a few learnable visual prompts to efficiently fine-tune the frozen pre-trained foundation model for downstream tasks, thereby accelerating training and improving model performance under limited local resources and data heterogeneity. Moreover, we design a global logit distillation scheme to handle the model heterogeneity and guide the local training. In detail, we leverage logits to implicitly capture local knowledge and design a weighted knowledge aggregation mechanism to generate global client-specific logits. We provide a theoretical guarantee on the generalization error bound for FedHPL. The experiments on various benchmark datasets under diverse settings of models and data demonstrate that our framework outperforms state-of-the-art FL approaches, with less computation overhead and training rounds.

翻译：联邦学习（FL）是一种流行的隐私保护范式，它使得分布式客户端能够在保持原始数据本地化的同时，与中央服务器协作训练模型。在实践中，本地客户端之间不同的模型架构、各异的数据分布以及有限的资源，不可避免地会导致模型性能下降和收敛速度减慢。然而，现有的联邦学习方法只能解决上述异构挑战中的部分问题，并且存在明显的性能局限。值得注意的是，目前尚未探索出一个统一的框架来克服这些挑战。为此，我们提出了FedHPL，一个基于**提示调优**与**对数蒸馏**的、面向**异构**设置的参数高效统一**联邦**学习框架。具体而言，我们采用一种本地提示调优方案，该方案利用少量可学习的视觉提示来高效微调冻结的预训练基础模型以适应下游任务，从而在有限的本地资源和数据异构性下加速训练并提升模型性能。此外，我们设计了一个全局对数蒸馏方案来处理模型异构性并指导本地训练。具体来说，我们利用对数来隐式捕获本地知识，并设计了一种加权知识聚合机制来生成全局的、客户端特定的对数。我们为FedHPL提供了泛化误差界的理论保证。在多种模型和数据设置下，于多个基准数据集上的实验表明，我们的框架优于最先进的联邦学习方法，同时具有更少的计算开销和训练轮数。