In several practical applications of federated learning (FL), the clients are highly heterogeneous in terms of both their data and compute resources, and therefore enforcing the same model architecture for each client is very limiting. Moreover, the need for uncertainty quantification and data privacy constraints are often particularly amplified for clients that have limited local data. This paper presents a unified FL framework to simultaneously address all these constraints and concerns, based on training customized local Bayesian models that learn well even in the absence of large local datasets. A Bayesian framework provides a natural way of incorporating supervision in the form of prior distributions. We use priors in the functional (output) space of the networks to facilitate collaboration across heterogeneous clients. Moreover, formal differential privacy guarantees are provided for this framework. Experiments on standard FL datasets demonstrate that our approach outperforms strong baselines in both homogeneous and heterogeneous settings and under strict privacy constraints, while also providing characterizations of model uncertainties.

翻译：在联邦学习的若干实际应用中，客户端在数据和计算资源方面高度异构，因此为每个客户端强制采用相同的模型架构具有很大局限性。此外，对于本地数据有限的客户端，不确定性量化与数据隐私约束的需求往往尤为突出。本文提出了一种统一的联邦学习框架，通过训练定制化的本地贝叶斯模型（即使缺乏大规模本地数据集也能良好学习），同时解决上述所有约束与问题。贝叶斯框架提供了一种以先验分布形式融入监督信息的自然方式。我们在网络的功能（输出）空间中使用先验，以促进异构客户端之间的协作。此外，该框架还提供了正式的差分隐私保证。在标准联邦学习数据集上的实验表明，我们的方法在同构与异构设置下、在严格隐私约束下均优于强基线方法，同时还能表征模型的不确定性。