Federated learning (FL) is a distributed learning framework where numerous clients collaborate with a central server to train a model without sharing local data. However, the standard federated optimization in real-world applications faces both statistical and system heterogeneity challenges, which result in unfavorable convergence behavior. The previous works attempted to modify the local training process (client-side) to tackle heterogeneity challenges. However, they ignored that the updates on the server side can coordinate the diverse local updates efficiently. This work explores the effect of server-side updates against heterogeneity issues. We first introduce the gradient diversity maximization direction findings, suggesting the global model moves continuously in this direction for fast and stable convergence. Then, we derive a novel server-side optimizer \textsc{FedAWARE} with rigorous convergence analysis for general non-convex settings. Our extensive experiments across multiple heterogeneous federated settings using four datasets showcase that \textsc{FedAWARE} achieves competitive convergence performance in comparison to state-of-the-art adaptive federated optimizers. Furthermore, our results show that \textsc{FedAWARE} can enhance the performance of FL algorithms as a plug-in module. Our source code is available at \url{https://github.com/dunzeng/FedAWARE}.

翻译：联邦学习（Federated Learning, FL）是一种分布式学习框架，众多客户端与中央服务器协作训练模型而无需共享本地数据。然而，实际应用中的标准联邦优化面临统计异构性和系统异构性挑战，导致收敛行为不佳。先前的研究尝试通过修改本地训练过程（客户端）来应对异构性挑战，但忽视了服务端更新可以有效协调多样化的本地更新。本文探讨了服务端更新对异构性问题的调控作用。我们首先介绍了梯度多样性最大化方向的发现，表明全局模型沿此方向持续移动可实现快速稳定的收敛。随后，我们推导出一种新颖的服务端优化器 \textsc{FedAWARE}，并为一般非凸场景提供了严格的收敛性分析。我们在四个数据集上进行的多组异构联邦学习场景实验表明，与最先进的自适应联邦优化器相比，\textsc{FedAWARE} 实现了具有竞争力的收敛性能。此外，实验结果显示 \textsc{FedAWARE} 可作为插件模块有效提升联邦学习算法的性能。源代码已公开于 \url{https://github.com/dunzeng/FedAWARE}。