Conventional federated learning algorithms train a single global model by leveraging all participating clients' data. However, due to heterogeneity in client generative distributions and predictive models, these approaches may not appropriately approximate the predictive process, converge to an optimal state, or generalize to new clients. We study personalization and generalization in stateless cross-device federated learning setups assuming heterogeneity in client data distributions and predictive models. We first propose a hierarchical generative model and formalize it using Bayesian Inference. We then approximate this process using Variational Inference to train our model efficiently. We call this algorithm Federated Variational Inference (FedVI). We use PAC-Bayes analysis to provide generalization bounds for FedVI. We evaluate our model on FEMNIST and CIFAR-100 image classification and show that FedVI beats the state-of-the-art on both tasks.

翻译：传统的联邦学习算法通过利用所有参与客户端的数据训练单一全局模型。然而，由于客户端生成分布及预测模型的异质性，这些方法可能无法恰当逼近预测过程、收敛至最优状态或泛化至新客户端。本文针对无状态跨设备联邦学习场景，假设客户端数据分布与预测模型存在异质性，研究了个性化与泛化问题。我们首先提出一个分层生成模型，并利用贝叶斯推断对其进行形式化建模；随后通过变分推断近似该过程以实现模型高效训练，将所提算法命名为联邦变分推断（FedVI）。我们采用PAC-Bayes分析为FedVI提供泛化界，并在FEMNIST和CIFAR-100图像分类任务上评估模型性能。实验表明，FedVI在两项任务中均超越了现有最优方法。