Data heterogeneity, characterized by disparities in local data distribution across clients, poses a significant challenge in federated learning. Substantial efforts have been devoted to addressing the heterogeneity in local label distribution. As minority classes suffer from worse accuracy due to overfitting on local imbalanced data, prior methods often incorporate class-balanced learning techniques during local training. Despite the improved mean accuracy across all classes, we observe that empty classes-referring to categories absent from a client's data distribution-are still not well recognized. This paper introduces FedED, a novel approach in heterogeneous federated learning that integrates both empty-class distillation and logit suppression simultaneously. Specifically, empty-class distillation leverages knowledge distillation during local training on each client to retain essential information related to empty classes from the global model. Moreover, logit suppression directly penalizes network logits for non-label classes, effectively addressing misclassifications in minority classes that may be biased toward majority classes. Extensive experiments validate the efficacy of FedED, surpassing previous state-of-the-art methods across diverse datasets with varying degrees of label distribution shift.

翻译：数据异构性，即客户端间局部数据分布的差异，是联邦学习中的一项重大挑战。大量研究致力于解决局部标签分布的异构性问题。由于少数类在局部不平衡数据上容易过拟合而导致准确率较低，以往方法通常会在局部训练中引入类别平衡学习技术。尽管所有类别的平均准确率得到提升，但我们观察到空类——即客户端数据分布中缺失的类别——仍未能得到良好识别。本文提出FedED，一种新颖的异构联邦学习方法，同时整合了空类蒸馏与logit抑制。具体而言，空类蒸馏在每个客户端的局部训练中利用知识蒸馏，从全局模型保留与空类相关的关键信息。此外，logit抑制直接惩罚非标签类别的网络logit，有效解决少数类可能偏向多数类的误分类问题。大量实验验证了FedED的有效性，在具有不同程度标签分布偏移的多种数据集上，其性能超越先前的最先进方法。