In federated learning (FL), a cluster of local clients are chaired under the coordination of the global server and cooperatively train one model with privacy protection. Due to the multiple local updates and the isolated non-iid dataset, clients are prone to overfit into their own optima, which extremely deviates from the global objective and significantly undermines the performance. Most previous works only focus on enhancing the consistency between the local and global objectives to alleviate this prejudicial client drifts from the perspective of the optimization view, whose performance would be prominently deteriorated on the high heterogeneity. In this work, we propose a novel and general algorithm {\ttfamily FedSMOO} by jointly considering the optimization and generalization targets to efficiently improve the performance in FL. Concretely, {\ttfamily FedSMOO} adopts a dynamic regularizer to guarantee the local optima towards the global objective, which is meanwhile revised by the global Sharpness Aware Minimization (SAM) optimizer to search for the consistent flat minima. Our theoretical analysis indicates that {\ttfamily FedSMOO} achieves fast $\mathcal{O}(1/T)$ convergence rate with low generalization bound. Extensive numerical studies are conducted on the real-world dataset to verify its peerless efficiency and excellent generality.

翻译：在联邦学习（FL）中，一组本地客户端在全局服务器的协调下，协作训练一个具有隐私保护的模型。由于多重本地更新和孤立的非独立同分布数据集，客户端容易过拟合到自身的局部最优解，这严重偏离全局目标并显著降低性能。以往大多数研究仅从优化视角关注增强局部与全局目标之间的一致性，以缓解这种有偏的客户端漂移，但在高度异构性下其性能会显著恶化。本文中，我们通过联合考虑优化和泛化目标，提出一种新颖且通用的算法FedSMOO，以有效提升FL的性能。具体而言，FedSMOO采用动态正则化器确保局部最优解朝向全局目标，同时通过全局尖锐感知最小化（SAM）优化器进行修正，以搜索一致的平坦极小值。我们的理论分析表明，FedSMOO实现了快速O(1/T)收敛速率和低泛化界。在真实世界数据集上进行了大量数值研究，验证了其卓越的效率和优秀的泛化能力。