In traditional federated learning, a single global model cannot perform equally well for all clients. Therefore, the need to achieve the client-level fairness in federated system has been emphasized, which can be realized by modifying the static aggregation scheme for updating the global model to an adaptive one, in response to the local signals of the participating clients. Our work reveals that existing fairness-aware aggregation strategies can be unified into an online convex optimization framework, in other words, a central server's sequential decision making process. To enhance the decision making capability, we propose simple and intuitive improvements for suboptimal designs within existing methods, presenting AAggFF. Considering practical requirements, we further subdivide our method tailored for the cross-device and the cross-silo settings, respectively. Theoretical analyses guarantee sublinear regret upper bounds for both settings: $\mathcal{O}(\sqrt{T \log{K}})$ for the cross-device setting, and $\mathcal{O}(K \log{T})$ for the cross-silo setting, with $K$ clients and $T$ federation rounds. Extensive experiments demonstrate that the federated system equipped with AAggFF achieves better degree of client-level fairness than existing methods in both practical settings. Code is available at https://github.com/vaseline555/AAggFF
翻译:在传统联邦学习中,单个全局模型无法对所有客户端均表现出同等性能。因此,联邦系统需要实现客户端层面的公平性,这可通过将静态的全局模型更新聚合方案修改为自适应方案来实现,从而响应参与客户端的本地信号。我们的研究表明,现有公平感知聚合策略可统一于在线凸优化框架,即中央服务器的序贯决策过程。为提升决策能力,我们针对现有方法中的次优设计提出了简单直观的改进方案,即AAggFF。考虑到实际需求,我们进一步将方法细分为适用于跨设备与跨孤岛两种场景的定制版本。理论分析为两种场景均提供了次线性遗憾上界保证:跨设备场景为$\mathcal{O}(\sqrt{T \log{K}})$,跨孤岛场景为$\mathcal{O}(K \log{T})$,其中$K$为客户端数量,$T$为联邦训练轮数。大量实验表明,配备AAggFF的联邦系统在两种实际场景中均能比现有方法实现更高程度的客户端层面公平性。代码发布于https://github.com/vaseline555/AAggFF