Federated Learning (FL) is a rising approach towards collaborative and privacy-preserving machine learning where large-scale medical datasets remain localized to each client. However, the issue of data heterogeneity among clients often compels local models to diverge, leading to suboptimal global models. To mitigate the impact of data heterogeneity on FL performance, we start with analyzing how FL training influence FL performance by decomposing the global loss into three terms: local loss, distribution shift loss and aggregation loss. Remarkably, our loss decomposition reveals that existing local training-based FL methods attempt to reduce the distribution shift loss, while the global aggregation-based FL methods propose better aggregation strategies to reduce the aggregation loss. Nevertheless, a comprehensive joint effort to minimize all three terms is currently limited in the literature, leading to subpar performance when dealing with data heterogeneity challenges. To fill this gap, we propose a novel FL method based on global loss decomposition, called FedLD, to jointly reduce these three loss terms. Our FedLD involves a margin control regularization in local training to reduce the distribution shift loss, and a principal gradient-based server aggregation strategy to reduce the aggregation loss. Notably, under different levels of data heterogeneity, our strategies achieve better and more robust performance on retinal and chest X-ray classification compared to other FL algorithms. Our code is available at https://github.com/Zeng-Shuang/FedLD.

翻译：联邦学习（Federated Learning, FL）是一种新兴的协作式隐私保护机器学习方法，其中大规模医学数据集仍保留在各客户端本地。然而，客户端间的数据异质性问题常导致本地模型发散，进而产生次优的全局模型。为减轻数据异质性对联邦学习性能的影响，我们首先通过将全局损失分解为三项——本地损失、分布偏移损失与聚合损失——来分析联邦学习训练如何影响其性能。值得注意的是，我们的损失分解表明：现有基于本地训练的联邦学习方法致力于降低分布偏移损失，而基于全局聚合的联邦学习方法则通过改进聚合策略来降低聚合损失。然而，现有文献中尚缺乏同时最小化这三项损失的综合性联合优化方案，导致应对数据异质性挑战时性能欠佳。为填补这一空白，我们提出一种基于全局损失分解的新型联邦学习方法——FedLD，以协同降低这三类损失项。FedLD在本地训练中引入边界控制正则化来降低分布偏移损失，并在服务器端采用基于主梯度的聚合策略以减少聚合损失。值得注意的是，在不同程度的数据异质性条件下，我们的方法在视网膜与胸部X光分类任务中相比其他联邦学习算法取得了更优且更稳健的性能。代码已开源：https://github.com/Zeng-Shuang/FedLD。