Bayesian personalized federated learning (BPFL) addresses challenges in existing personalized FL (PFL). BPFL aims to quantify the uncertainty and heterogeneity within and across clients towards uncertainty representations by addressing the statistical heterogeneity of client data. In PFL, some recent preliminary work proposes to decompose hidden neural representations into shared and local components and demonstrates interesting results. However, most of them do not address client uncertainty and heterogeneity in FL systems, while appropriately decoupling neural representations is challenging and often ad hoc. In this paper, we make the first attempt to introduce a general BPFL framework to decompose and jointly learn shared and personalized uncertainty representations on statistically heterogeneous client data over time. A Bayesian federated neural network BPFed instantiates BPFL by jointly learning cross-client shared uncertainty and client-specific personalized uncertainty over statistically heterogeneous and randomly participating clients. We further involve continual updating of prior distribution in BPFed to speed up the convergence and avoid catastrophic forgetting. Theoretical analysis and guarantees are provided in addition to the experimental evaluation of BPFed against the diversified baselines.

翻译：贝叶斯个性化联邦学习（BPFL）解决了现有个性化联邦学习（PFL）中的挑战。BPFL旨在通过处理客户端数据的统计异质性，量化客户端内部及客户端间关于不确定性表示的异质性与不确定性。在PFL中，近期一些初步工作提出将隐藏神经表示分解为共享组件和局部组件，并展示了有趣的结果。然而，这些工作大多未解决联邦学习系统中客户端的不确定性与异质性，且神经表示的恰当解耦具有挑战性且常带有临时性。本文首次尝试引入一个通用BPFL框架，在统计异构的客户端数据随时间变化的过程中，分解并联合学习共享与个性化不确定性表示。贝叶斯联邦神经网络BPFed通过联合学习统计异构且随机参与客户端的跨客户端共享不确定性及客户端特定个性化不确定性，实例化BPFL。我们进一步在BPFed中引入先验分布的连续更新，以加速收敛并避免灾难性遗忘。除对BPFed与多样化基线进行实验评估外，本文还提供了理论分析与保证。