Standard federated learning (FL) algorithms typically require multiple rounds of communication between the server and the clients, which has several drawbacks, including requiring constant network connectivity, repeated investment of computational resources, and susceptibility to privacy attacks. One-Shot FL is a new paradigm that aims to address this challenge by enabling the server to train a global model in a single round of communication. In this work, we present FedFisher, a novel algorithm for one-shot FL that makes use of Fisher information matrices computed on local client models, motivated by a Bayesian perspective of FL. First, we theoretically analyze FedFisher for two-layer over-parameterized ReLU neural networks and show that the error of our one-shot FedFisher global model becomes vanishingly small as the width of the neural networks and amount of local training at clients increases. Next, we propose practical variants of FedFisher using the diagonal Fisher and K-FAC approximation for the full Fisher and highlight their communication and compute efficiency for FL. Finally, we conduct extensive experiments on various datasets, which show that these variants of FedFisher consistently improve over competing baselines.

翻译：标准联邦学习算法通常需要在服务器与客户端之间进行多轮通信，这存在多个缺点，包括需要持续的网络连接、重复投入计算资源以及易受隐私攻击。一次性联邦学习是一种新范式，旨在通过使服务器在单轮通信中训练全局模型来应对这一挑战。在本工作中，我们提出FedFisher——一种基于贝叶斯视角的联邦学习算法，利用在本地客户端模型上计算的Fisher信息矩阵。首先，我们从理论上分析了用于两层过参数化ReLU神经网络的FedFisher，并证明随着神经网络宽度和客户端本地训练量的增加，我们的一次性FedFisher全局模型的误差趋近于零。其次，我们提出了使用对角Fisher和K-FAC近似完整Fisher的FedFisher实用变体，并强调了其在联邦学习中的通信和计算效率。最后，我们在多个数据集上进行了大量实验，结果表明这些FedFisher变体持续优于竞争基线方法。