In federated learning (FL), the significant communication overhead due to the slow convergence speed of training the global model poses a great challenge. Specifically, a large number of communication rounds are required to achieve the convergence in FL. One potential solution is to employ the Newton-based optimization method for training, known for its quadratic convergence rate. However, the existing Newton-based FL training methods suffer from either memory inefficiency or high computational costs for local clients or the server. To address this issue, we propose an FL with approximated global Hessian (FAGH) method to accelerate FL training. FAGH leverages the first moment of the approximated global Hessian and the first moment of the global gradient to train the global model. By harnessing the approximated global Hessian curvature, FAGH accelerates the convergence of global model training, leading to the reduced number of communication rounds and thus the shortened training time. Experimental results verify FAGH's effectiveness in decreasing the number of communication rounds and the time required to achieve the pre-specified objectives of the global model performance in terms of training and test losses as well as test accuracy. Notably, FAGH outperforms several state-of-the-art FL training methods.

翻译：在联邦学习（FL）中，由于全局模型训练收敛速度缓慢导致的显著通信开销构成了巨大挑战。具体而言，FL需要大量通信轮次才能实现收敛。一种潜在的解决方案是采用具有二次收敛速率的牛顿优化方法进行训练。然而，现有的基于牛顿法的FL训练方法存在内存效率低下或本地客户端/服务器端计算成本高昂的问题。为解决该问题，我们提出了一种基于近似全局Hessian矩阵的联邦学习方法（FAGH）以加速FL训练。FAGH利用近似全局Hessian矩阵的一阶矩与全局梯度的一阶矩来训练全局模型。通过利用近似全局Hessian曲率，FAGH加速了全局模型训练的收敛过程，从而减少了通信轮次并缩短了训练时间。实验结果验证了FAGH在减少通信轮次以及缩短达到预设全局模型性能目标（包括训练损失、测试损失及测试准确率）所需时间方面的有效性。值得注意的是，FAGH在性能上超越了多种最先进的FL训练方法。