Federated Learning (FL) has surged in prominence due to its capability of collaborative model training without direct data sharing. However, the vast disparity in local data distributions among clients, often termed the non-Independent Identically Distributed (non-IID) challenge, poses a significant hurdle to FL's generalization efficacy. The scenario becomes even more complex when not all clients participate in the training process, a common occurrence due to unstable network connections or limited computational capacities. This can greatly complicate the assessment of the trained models' generalization abilities. While a plethora of recent studies has centered on the generalization gap pertaining to unseen data from participating clients with diverse distributions, the divergence between the training distributions of participating clients and the testing distributions of non-participating ones has been largely overlooked. In response, our paper unveils an information-theoretic generalization framework for FL. Specifically, it quantifies generalization errors by evaluating the information entropy of local distributions and discerning discrepancies across these distributions. Inspired by our deduced generalization bounds, we introduce a weighted aggregation approach and a duo of client selection strategies. These innovations aim to bolster FL's generalization prowess by encompassing a more varied set of client data distributions. Our extensive empirical evaluations reaffirm the potency of our proposed methods, aligning seamlessly with our theoretical construct.

翻译：联邦学习因能在不直接共享数据的情况下协同训练模型而备受关注。然而，客户端间本地数据分布的显著差异（即非独立同分布难题）严重制约了联邦学习的泛化能力。当部分客户端因网络连接不稳定或计算能力限制而无法参与训练时（这是常见场景），情况将更为复杂，这极大增加了训练模型泛化性能评估的难度。现有研究多聚焦于参与客户端中不同分布数据对应的未见过样本的泛化差距，但鲜有考虑参与客户端训练分布与非参与客户端测试分布之间的差异。为此，本文提出了一种基于信息论的联邦学习泛化框架：通过评估本地分布的信息熵并识别这些分布间的差异来量化泛化误差。基于推导的泛化界，我们进一步提出了加权聚合策略与两种客户端选择方案。这些创新方法通过覆盖更多样化的客户端数据分布来增强联邦学习的泛化能力。大量实验验证了我们方法的有效性，其结果与理论框架高度吻合。