Label skews, characterized by disparities in local label distribution across clients, pose a significant challenge in federated learning. As minority classes suffer from worse accuracy due to overfitting on local imbalanced data, prior methods often incorporate class-balanced learning techniques during local training. Although these methods improve the mean accuracy across all classes, we observe that vacant classes-referring to categories absent from a client's data distribution-remain poorly recognized. Besides, there is still a gap in the accuracy of local models on minority classes compared to the global model. This paper introduces FedVLS, a novel approach to label-skewed federated learning that integrates both vacant-class distillation and logit suppression simultaneously. Specifically, vacant-class distillation leverages knowledge distillation during local training on each client to retain essential information related to vacant classes from the global model. Moreover, logit suppression directly penalizes network logits for non-label classes, effectively addressing misclassifications in minority classes that may be biased toward majority classes. Extensive experiments validate the efficacy of FedVLS, demonstrating superior performance compared to previous state-of-the-art (SOTA) methods across diverse datasets with varying degrees of label skews. Our code is available at https://github.com/krumpguo/FedVLS.

翻译：标签偏斜，即客户端间本地标签分布存在差异，是联邦学习中的一个重大挑战。由于少数类别在本地不平衡数据上容易过拟合而导致准确率下降，现有方法通常在本地训练中引入类别平衡学习技术。尽管这些方法提升了所有类别的平均准确率，但我们观察到，空缺类别——即客户端数据分布中缺失的类别——的识别效果仍然很差。此外，本地模型在少数类别上的准确率与全局模型相比仍存在差距。本文提出FedVLS，一种新颖的标签偏斜联邦学习方法，它同时集成了空缺类别蒸馏与对数抑制。具体而言，空缺类别蒸馏在每个客户端的本地训练过程中利用知识蒸馏，以保留来自全局模型的、与空缺类别相关的关键信息。此外，对数抑制直接对非标签类别的网络输出对数施加惩罚，有效解决了少数类别可能被偏向多数类别的错误分类问题。大量实验验证了FedVLS的有效性，其在具有不同程度标签偏斜的多种数据集上均表现出优于先前最先进（SOTA）方法的性能。我们的代码发布于 https://github.com/krumpguo/FedVLS。