This paper proposes a novel federated algorithm that leverages momentum-based variance reduction with adaptive learning to address non-convex settings across heterogeneous data. We intend to minimize communication and computation overhead, thereby fostering a sustainable federated learning system. We aim to overcome challenges related to gradient variance, which hinders the model's efficiency, and the slow convergence resulting from learning rate adjustments with heterogeneous data. The experimental results on the image classification tasks with heterogeneous data reveal the effectiveness of our suggested algorithms in non-convex settings with an improved communication complexity of $\mathcal{O}(\epsilon^{-1})$ to converge to an $\epsilon$-stationary point - compared to the existing communication complexity $\mathcal{O}(\epsilon^{-2})$ of most prior works. The proposed federated version maintains the trade-off between the convergence rate, number of communication rounds, and test accuracy while mitigating the client drift in heterogeneous settings. The experimental results demonstrate the efficiency of our algorithms in image classification tasks (MNIST, CIFAR-10) with heterogeneous data.

翻译：本文提出一种新颖的联邦算法，该算法通过结合动量式方差缩减与自适应学习机制，以解决异构数据下的非凸优化问题。我们致力于最小化通信与计算开销，从而构建可持续的联邦学习系统。本方法旨在克服梯度方差对模型效率的制约，以及异构数据环境下因学习率调整导致的收敛速度缓慢问题。在异构数据图像分类任务上的实验结果表明，所提算法在非凸场景中具有显著有效性：为收敛至$\epsilon$-平稳点，其通信复杂度提升至$\mathcal{O}(\epsilon^{-1})$，而现有多数工作的通信复杂度为$\mathcal{O}(\epsilon^{-2})$。所提出的联邦版本在收敛速率、通信轮数与测试精度之间保持了平衡，同时缓解了异构环境下的客户端漂移现象。在异构数据图像分类任务（MNIST、CIFAR-10）上的实验结果验证了算法的优越性能。