In this work, we propose GPT-FL, a generative pre-trained model-assisted federated learning (FL) framework. At its core, GPT-FL leverages generative pre-trained models to generate diversified synthetic data. These generated data are used to train a downstream model on the server, which is then fine-tuned with private client data under the standard FL framework. We show that GPT-FL consistently outperforms state-of-the-art FL methods in terms of model test accuracy, communication efficiency, and client sampling efficiency. Through comprehensive ablation analysis across various data modalities, we discover that the downstream model generated by synthetic data plays a crucial role in controlling the direction of gradient diversity during FL training, which enhances convergence speed and contributes to the notable accuracy boost observed with GPT-FL. Also, regardless of whether the target data falls within or outside the domain of the pre-trained generative model, GPT-FL consistently achieves significant performance gains, surpassing the results obtained by models trained solely with FL or synthetic data. The code is available at https://github.com/AvestimehrResearchGroup/GPT-FL.

翻译：在本工作中，我们提出了GPT-FL，一种生成式预训练模型辅助的联邦学习（FL）框架。其核心在于利用生成式预训练模型生成多样化的合成数据。这些生成的数据用于在服务器端训练一个下游模型，随后该模型在标准FL框架下使用私有客户端数据进行微调。我们证明，GPT-FL在模型测试精度、通信效率和客户端采样效率方面均持续优于最先进的FL方法。通过对多种数据模态的综合消融分析，我们发现由合成数据生成的下游模型在控制FL训练期间梯度多样性的方向上起着关键作用，这提升了收敛速度，并有助于实现GPT-FL所观察到的显著精度提升。此外，无论目标数据是否处于预训练生成模型的领域内，GPT-FL均能持续取得显著的性能增益，超越仅使用FL或合成数据训练的模型所获得的结果。代码可在 https://github.com/AvestimehrResearchGroup/GPT-FL 获取。