Federated learning (FL) collaboratively trains artificial intelligence (AI) models to ensure user data privacy. Sharing only model updates generated from local training on client data with the server enhances user data privacy. However, model performance may suffer due to data and system heterogeneity among clients in FL scenarios. Previous studies have proposed model optimization, fine-tuning, and personalization to achieve improved model performance. Despite these efforts, models resulting from FL scenarios often exhibit catastrophic forgetting, which increases the communication and computational costs of clients for model optimization and raises energy consumption. To address these challenges, we propose a reference model-based fine-tuning method for federated learning that overcomes catastrophic forgetting in each round. Our method is derived from Bayesian parameter-efficient transfer learning and includes an proximal term. It employs a reference model that incorporates previous model parameters and reviews previous global features in the model optimization step to mitigate catastrophic forgetting. As a result, our method achieves higher model performance and lower communication and computational costs for clients than existing methods.

翻译：联邦学习（FL）通过协作训练人工智能（AI）模型以确保用户数据隐私。仅将客户端本地训练生成的模型更新共享至服务器，增强了用户数据隐私保护。然而，在联邦学习场景中，由于客户端间的数据与系统异质性，模型性能可能受到影响。先前研究提出了模型优化、微调及个性化方法以提升模型性能。尽管已有这些努力，联邦学习场景中产生的模型仍常出现灾难性遗忘现象，这增加了客户端模型优化的通信与计算成本，并导致能耗上升。为应对这些挑战，我们提出一种基于参考模型的联邦学习微调方法，该方法在每轮训练中克服灾难性遗忘。我们的方法源自贝叶斯参数高效迁移学习，并包含一个近端项。该方法采用一个融合历史模型参数的参考模型，在模型优化步骤中回顾先前的全局特征，以缓解灾难性遗忘。因此，与现有方法相比，我们的方法能为客户端实现更高的模型性能以及更低的通信与计算成本。