Ensuring fairness in a Federated Learning (FL) system, i.e., a satisfactory performance for all of the participating diverse clients, is an important and challenging problem. There are multiple fair FL algorithms in the literature, which have been relatively successful in providing fairness. However, these algorithms mostly emphasize on the loss functions of worst-off clients to improve their performance, which often results in the suppression of well-performing ones. As a consequence, they usually sacrifice the system's overall average performance for achieving fairness. Motivated by this and inspired by two well-known risk modeling methods in Finance, Mean-Variance and Mean-Semi-Variance, we propose and study two new fair FL algorithms, Variance Reduction (VRed) and Semi-Variance Reduction (SemiVRed). VRed encourages equality between clients' loss functions by penalizing their variance. In contrast, SemiVRed penalizes the discrepancy of only the worst-off clients' loss functions from the average loss. Through extensive experiments on multiple vision and language datasets, we show that, SemiVRed achieves SoTA performance in scenarios with heterogeneous data distributions and improves both fairness and system overall average performance.

翻译：确保联邦学习（FL）系统的公平性——即所有参与其中的多样化客户端均获得令人满意的性能——是一个重要且具有挑战性的问题。现有文献中已提出多种公平联邦学习算法，这些算法在提供公平性方面取得了一定成功。然而，这些算法大多侧重于改善最弱势客户端的损失函数以提升其性能，这往往会导致对表现良好客户端的抑制。因此，它们通常以牺牲系统整体平均性能为代价来实现公平。受此启发，并借鉴金融领域中两种著名的风险建模方法——均值-方差与均值-半方差，我们提出并研究了两种新的公平联邦学习算法：方差缩减（VRed）与半方差缩减（SemiVRed）。VRed通过惩罚客户端损失函数的方差来促进损失函数间的均衡。相比之下，SemiVRed仅惩罚最弱势客户端损失函数与平均损失之间的差异。通过在多个视觉与语言数据集上的大量实验，我们证明SemiVRed在数据分布异构的场景下取得了当前最优（SoTA）性能，同时提升了公平性与系统整体平均性能。