Federated Learning (FL) has surged in prominence due to its capability of collaborative model training without direct data sharing. However, the vast disparity in local data distributions among clients, often termed the Non-Independent Identically Distributed (Non-IID) challenge, poses a significant hurdle to FL's generalization efficacy. The scenario becomes even more complex when not all clients participate in the training process, a common occurrence due to unstable network connections or limited computational capacities. This can greatly complicate the assessment of the trained models' generalization abilities. While a plethora of recent studies has centered on the generalization gap pertaining to unseen data from participating clients with diverse distributions, the distinction between the training distributions of participating clients and the testing distributions of non-participating ones has been largely overlooked. In response, our paper unveils an information-theoretic generalization framework for FL. Specifically, it quantifies generalization errors by evaluating the information entropy of local distributions and discerning discrepancies across these distributions. Inspired by our deduced generalization bounds, we introduce a weighted aggregation approach and a duo of client selection strategies. These innovations are designed to strengthen FL's ability to generalize and thus ensure that trained models perform better on non-participating clients by incorporating a more diverse range of client data distributions. Our extensive empirical evaluations reaffirm the potency of our proposed methods, aligning seamlessly with our theoretical construct.

翻译：联邦学习（Federated Learning, FL）因其能够在无需直接共享数据的情况下实现协同模型训练而备受关注。然而，客户端本地数据分布间存在的巨大差异——常被称为非独立同分布（Non-IID）挑战——对联邦学习的泛化效能构成了显著障碍。当并非所有客户端都参与训练过程时，情况变得更为复杂，这通常由不稳定的网络连接或有限的计算能力导致。这会极大地增加评估已训练模型泛化能力的复杂性。尽管近期大量研究集中于关注来自具有不同分布的参与客户端的未见数据的泛化差距，但参与客户端的训练分布与非参与客户端的测试分布之间的差异在很大程度上被忽视了。为此，本文提出了一个面向联邦学习的信息论泛化框架。具体而言，该框架通过评估本地分布的信息熵并识别这些分布间的差异来量化泛化误差。受我们推导出的泛化界启发，我们引入了一种加权聚合方法以及一对客户端选择策略。这些创新旨在增强联邦学习的泛化能力，从而通过纳入更多样化的客户端数据分布，确保训练后的模型在非参与客户端上表现更优。我们广泛的实证评估再次证实了所提方法的有效性，其结果与我们的理论构建完全吻合。