Optimization-based techniques for federated learning (FL) often come with prohibitive communication cost, as high dimensional model parameters need to be communicated repeatedly between server and clients. In this paper, we follow a Bayesian approach allowing to perform FL with one-shot communication, by solving the global inference problem as a product of local client posteriors. For models with multi-modal likelihoods, such as neural networks, a naive application of this scheme is hampered, since clients will capture different posterior modes, causing a destructive collapse of the posterior on the server side. Consequently, we explore approximate inference in the function-space representation of client posteriors, hence suffering less or not at all from multi-modality. We show that distributed function-space inference is tightly related to learning Bayesian pseudocoresets and develop a tractable Bayesian FL algorithm on this insight. We show that this approach achieves prediction performance competitive to state-of-the-art while showing a striking reduction in communication cost of up to two orders of magnitude. Moreover, due to its Bayesian nature, our method also delivers well-calibrated uncertainty estimates.

翻译：基于优化的联邦学习技术通常伴随着高昂的通信开销，因为高维模型参数需要在服务器与客户端之间反复传递。本文采用贝叶斯方法，通过将全局推断问题分解为局部客户端后验的乘积，实现了仅需单次通信的联邦学习。对于具有多模态似然函数的模型（例如神经网络），该方案的直接应用会受到阻碍，因为客户端将捕获不同的后验模态，导致服务器端后验发生破坏性崩塌。因此，我们在客户端后验的函数空间表示中探索近似推断，从而较少或完全不受多模态问题的影响。我们证明了分布式函数空间推断与学习贝叶斯伪核心集紧密相关，并基于这一洞见开发了一种可处理的贝叶斯联邦学习算法。实验表明，该方法在保持与最先进技术相当的预测性能的同时，通信开销显著降低了高达两个数量级。此外，得益于其贝叶斯特性，我们的方法还能提供校准良好的不确定性估计。