Federated learning (FL) allows multiple clients to collaboratively learn a globally shared model through cycles of model aggregation and local model training, without the need to share data. Most existing FL methods train local models separately on different clients, and then simply average their parameters to obtain a centralized model on the server side. However, these approaches generally suffer from large aggregation errors and severe local forgetting, which are particularly bad in heterogeneous data settings. To tackle these issues, in this paper, we propose a novel FL framework that uses online Laplace approximation to approximate posteriors on both the client and server side. On the server side, a multivariate Gaussian product mechanism is employed to construct and maximize a global posterior, largely reducing the aggregation errors induced by large discrepancies between local models. On the client side, a prior loss that uses the global posterior probabilistic parameters delivered from the server is designed to guide the local training. Binding such learning constraints from other clients enables our method to mitigate local forgetting. Finally, we achieve state-of-the-art results on several benchmarks, clearly demonstrating the advantages of the proposed method.

翻译：联邦学习（FL）允许多个客户端通过模型聚合与本地模型训练的循环过程协同学习一个全局共享模型，而无需共享数据。现有的大多数FL方法在不同客户端上分别训练本地模型，然后简单地对它们的参数取平均，以在服务器端获得集中式模型。然而，这些方法通常遭受较大的聚合误差和严重的局部遗忘，这在异质数据场景下尤为严重。为解决这些问题，本文提出了一种新颖的FL框架，该框架在客户端和服务器端均采用在线拉普拉斯近似来逼近后验分布。在服务器端，采用多元高斯乘积机制来构造并最大化全局后验，从而大幅减少由本地模型间较大差异引起的聚合误差。在客户端，设计了一种先验损失，利用服务器传递的全局后验概率参数来指导本地训练。通过绑定来自其他客户端的学习约束，我们的方法能够缓解局部遗忘。最终，我们在多个基准测试上取得了最先进的结果，清楚展示了所提方法的优势。