Federated Learning (FL) models often experience client drift caused by heterogeneous data, where the distribution of data differs across clients. To address this issue, advanced research primarily focuses on manipulating the existing gradients to achieve more consistent client models. In this paper, we present an alternative perspective on client drift and aim to mitigate it by generating improved local models. First, we analyze the generalization contribution of local training and conclude that this generalization contribution is bounded by the conditional Wasserstein distance between the data distribution of different clients. Then, we propose FedImpro, to construct similar conditional distributions for local training. Specifically, FedImpro decouples the model into high-level and low-level components, and trains the high-level portion on reconstructed feature distributions. This approach enhances the generalization contribution and reduces the dissimilarity of gradients in FL. Experimental results show that FedImpro can help FL defend against data heterogeneity and enhance the generalization performance of the model.

翻译：联邦学习（FL）模型常因数据异构性（不同客户端数据分布存在差异）导致客户端漂移。为解决该问题，现有研究主要通过操控现有梯度来实现更一致的客户端模型。本文提出客户端漂移的新视角，并尝试通过生成改进的局部模型来缓解该问题。首先，我们分析局部训练的泛化贡献，并指出该泛化贡献受不同客户端数据分布间条件Wasserstein距离的约束。进而，我们提出FedImpro方法，通过构建相似的局部训练条件分布来优化。具体而言，FedImpro将模型解耦为高层与低层组件，并在重构的特征分布上训练高层部分。该方法能增强泛化贡献、降低FL中梯度的差异性。实验结果表明，FedImpro可帮助FL抵御数据异构性，并提升模型的泛化性能。