Federated learning aims to infer a shared model from private and decentralized data stored locally by multiple clients. Personalized federated learning (PFL) goes one step further by adapting the global model to each client, enhancing the model's fit for different clients. A significant level of personalization is required for highly heterogeneous clients, but can be challenging to achieve especially when they have small datasets. To address this problem, we propose a PFL algorithm named PAC-PFL for learning probabilistic models within a PAC-Bayesian framework that utilizes differential privacy to handle data-dependent priors. Our algorithm collaboratively learns a shared hyper-posterior and regards each client's posterior inference as the personalization step. By establishing and minimizing a generalization bound on the average true risk of clients, PAC-PFL effectively combats over-fitting. PACPFL achieves accurate and well-calibrated predictions, supported by experiments on a dataset of photovoltaic panel power generation, FEMNIST dataset (Caldas et al., 2019), and Dirichlet-partitioned EMNIST dataset (Cohen et al., 2017).

翻译：联邦学习旨在从多个客户端本地存储的私有和分散化数据中推断共享模型。个性化联邦学习（PFL）通过将全局模型适配至每个客户端，进一步提升了模型对不同客户端的拟合效果。对于高度异构的客户端群体，需要显著的个性化程度，但在客户端数据集较小的情况下实现这一目标尤为困难。为解决该问题，我们提出一种名为PAC-PFL的PFL算法，用于在PAC-Bayesian框架内学习概率模型，该框架利用差分隐私处理数据相关的先验分布。我们的算法协同学习共享的超后验分布，并将每个客户端的后验推断视为个性化步骤。通过建立并最小化客户端平均真实风险上的泛化界，PAC-PFL有效缓解过拟合问题。在光伏面板发电数据集、FEMNIST数据集（Caldas等，2019）及Dirichlet划分的EMNIST数据集（Cohen等，2017）上的实验表明，PAC-PFL实现了精确且校准良好的预测。