Federated learning (FL) has emerged as a prominent approach for collaborative training of machine learning models across distributed clients while preserving data privacy. However, the quest to balance acceleration and stability becomes a significant challenge in FL, especially on the client-side. In this paper, we introduce FedCAda, an innovative federated client adaptive algorithm designed to tackle this challenge. FedCAda leverages the Adam algorithm to adjust the correction process of the first moment estimate $m$ and the second moment estimate $v$ on the client-side and aggregate adaptive algorithm parameters on the server-side, aiming to accelerate convergence speed and communication efficiency while ensuring stability and performance. Additionally, we investigate several algorithms incorporating different adjustment functions. This comparative analysis revealed that due to the limited information contained within client models from other clients during the initial stages of federated learning, more substantial constraints need to be imposed on the parameters of the adaptive algorithm. As federated learning progresses and clients gather more global information, FedCAda gradually diminishes the impact on adaptive parameters. These findings provide insights for enhancing the robustness and efficiency of algorithmic improvements. Through extensive experiments on computer vision (CV) and natural language processing (NLP) datasets, we demonstrate that FedCAda outperforms the state-of-the-art methods in terms of adaptability, convergence, stability, and overall performance. This work contributes to adaptive algorithms for federated learning, encouraging further exploration.

翻译：联邦学习（FL）已成为一种在分布式客户端间协同训练机器学习模型并保护数据隐私的重要方法。然而，在客户端侧实现加速与稳定性的平衡成为FL中的重大挑战。本文提出FedCAda——一种创新的联邦客户端自适应算法，旨在应对这一挑战。FedCAda利用Adam算法在客户端侧调整一阶矩估计$m$和二阶矩估计$v$的修正过程，并在服务器侧聚合自适应算法参数，以在确保稳定性和性能的同时加速收敛速度并提升通信效率。此外，我们研究了多种采用不同调整函数的算法。对比分析表明，由于联邦学习初期客户端模型包含的来自其他客户端的信息有限，需对自适应算法参数施加更强的约束。随着联邦学习推进及客户端获取更多全局信息，FedCAda逐步减弱对自适应参数的影响。这些发现为提升算法改进的鲁棒性和效率提供了启示。通过在计算机视觉（CV）和自然语言处理（NLP）数据集上的大量实验，我们证明FedCAda在适应性、收敛性、稳定性和整体性能方面均优于最先进方法。本工作为联邦学习自适应算法研究做出贡献，并鼓励进一步探索。