This paper addresses intra-client and inter-client covariate shifts in federated learning (FL) with a focus on the overall generalization performance. To handle covariate shifts, we formulate a new global model training paradigm and propose Federated Importance-Weighted Empirical Risk Minimization (FTW-ERM) along with improving density ratio matching methods without requiring perfect knowledge of the supremum over true ratios. We also propose the communication-efficient variant FITW-ERM with the same level of privacy guarantees as those of classical ERM in FL. We theoretically show that FTW-ERM achieves smaller generalization error than classical ERM under certain settings. Experimental results demonstrate the superiority of FTW-ERM over existing FL baselines in challenging imbalanced federated settings in terms of data distribution shifts across clients.

翻译：本文针对联邦学习中客户端内部及客户端间的协变量偏移问题，重点研究整体泛化性能的提升。为应对协变量偏移，我们构建了新的全局模型训练范式，提出联邦重要性加权经验风险最小化方法，并改进了密度比匹配技术，无需精确掌握真实比值的上确界。同时提出通信效率优化的变体方法，其隐私保护水平与联邦学习中经典经验风险最小化方法相当。理论分析表明，在特定设定下，所提方法比经典经验风险最小化方法具有更小的泛化误差。实验结果显示，在跨客户端数据分布偏移严重的非均衡联邦场景中，该方法显著优于现有联邦学习基线方法。