Federated learning (FL) aims to collaboratively train a shared model across multiple clients without transmitting their local data. Data heterogeneity is a critical challenge in realistic FL settings, as it causes significant performance deterioration due to discrepancies in optimization among local models. In this work, we focus on label distribution skew, a common scenario in data heterogeneity, where the data label categories are imbalanced on each client. To address this issue, we propose FedBalance, which corrects the optimization bias among local models by calibrating their logits. Specifically, we introduce an extra private weak learner on the client side, which forms an ensemble model with the local model. By fusing the logits of the two models, the private weak learner can capture the variance of different data, regardless of their category. Therefore, the optimization direction of local models can be improved by increasing the penalty for misclassifying minority classes and reducing the attention to majority classes, resulting in a better global model. Extensive experiments show that our method can gain 13\% higher average accuracy compared with state-of-the-art methods.

翻译：联邦学习旨在无需传输客户端本地数据的情况下，跨多个客户端协作训练共享模型。数据异构性是现实联邦学习场景中的关键挑战，由于本地模型间优化差异导致性能显著下降。本研究聚焦标签分布倾斜问题——数据异构性中常见场景，即各客户端数据标签类别分布不均衡。为应对该问题，我们提出FedBalance方法，通过校准logits修正本地模型间的优化偏差。具体而言，我们在客户端引入额外私有弱学习器，与本地模型构成集成模型。通过融合两个模型的logits，私有弱学习器能捕获不同数据的方差特征（不依赖其类别）。由此，通过增加对少数类误分类的惩罚并降低对多数类的关注，可优化本地模型的优化方向，从而获得更优的全局模型。大量实验表明，与当前最先进方法相比，本方法平均准确率提升13%。