Federated Learning (FL) aims to infer a shared model from private and decentralized data stored by multiple clients. Personalized FL (PFL) enhances the model's fit for each client by adapting the global model to the clients. A significant level of personalization is required for highly heterogeneous clients but can be challenging to achieve, especially when clients' datasets are small. To address this issue, we introduce the PAC-PFL framework for PFL of probabilistic models. PAC-PFL infers a shared hyper-posterior and treats each client's posterior inference as the personalization step. Unlike previous PFL algorithms, PAC-PFL does not regularize all personalized models towards a single shared model, thereby greatly enhancing its personalization flexibility. By establishing and minimizing a PAC-Bayesian generalization bound on the average true loss of clients, PAC-PFL effectively mitigates overfitting even in data-poor scenarios. Additionally, PAC-PFL provides generalization bounds for new clients joining later. PAC-PFL achieves accurate and well-calibrated predictions, as supported by our experiments.

翻译：联邦学习（FL）旨在从多个客户端存储的私有且分散的数据中推断出一个共享模型。个性化联邦学习（PFL）通过使全局模型适应各客户端来增强模型对每个客户端的拟合度。对于高度异构的客户端，需要显著程度的个性化，但这可能难以实现，尤其是在客户端数据集较小时。为解决此问题，我们提出了用于概率模型PFL的PAC-PFL框架。PAC-PFL推断一个共享的超后验，并将每个客户端的后验推断视为个性化步骤。与以往的PFL算法不同，PAC-PFL不将所有个性化模型正则化至单一共享模型，从而极大地增强了其个性化灵活性。通过建立并最小化客户端平均真实损失的PAC-Bayesian泛化界，PAC-PFL即使在数据匮乏的情况下也能有效缓解过拟合。此外，PAC-PFL为后续加入的新客户端提供了泛化界。我们的实验证实，PAC-PFL能够实现准确且校准良好的预测。