In Federated Learning (FL), a framework to train machine learning models across distributed data, well-known algorithms like FedAvg tend to have slow convergence rates, resulting in high communication costs during training. To address this challenge, we introduce FedLion, an adaptive federated optimization algorithm that seamlessly incorporates key elements from the recently proposed centralized adaptive algorithm, Lion (Chen et al. 2o23), into the FL framework. Through comprehensive evaluations on two widely adopted FL benchmarks, we demonstrate that FedLion outperforms previous state-of-the-art adaptive algorithms, including FAFED (Wu et al. 2023) and FedDA. Moreover, thanks to the use of signed gradients in local training, FedLion substantially reduces data transmission requirements during uplink communication when compared to existing adaptive algorithms, further reducing communication costs. Last but not least, this work also includes a novel theoretical analysis, showcasing that FedLion attains faster convergence rate than established FL algorithms like FedAvg.

翻译：在联邦学习（FL）这一分布式数据训练机器学习模型的框架中，诸如FedAvg等知名算法往往收敛速度较慢，导致训练过程中通信开销居高不下。为应对这一挑战，我们提出FedLion——一种自适应联邦优化算法，它将近期提出的集中式自适应算法Lion（Chen等人，2023）的核心要素无缝融入FL框架。通过在两个广泛使用的FL基准数据集上的全面评估，我们证明FedLion在性能上超越了此前最先进的自适应算法，包括FAFED（Wu等人，2023）和FedDA。此外，得益于局部训练中采用符号梯度，相较现有自适应算法，FedLion大幅降低了上行通信阶段的数据传输需求，从而进一步削减通信成本。最后，本文还包含一项创新的理论分析，表明FedLion比FedAvg等已有FL算法实现了更快的收敛速度。