Federated Learning (FL) has recently emerged as a popular framework, which allows resource-constrained discrete clients to cooperatively learn the global model under the orchestration of a central server while storing privacy-sensitive data locally. However, due to the difference in equipment and data divergence of heterogeneous clients, there will be parameter deviation between local models, resulting in a slow convergence rate and a reduction of the accuracy of the global model. The current FL algorithms use the static client learning strategy pervasively and can not adapt to the dynamic training parameters of different clients. In this paper, by considering the deviation between different local model parameters, we propose an adaptive learning rate scheme for each client based on entropy theory to alleviate the deviation between heterogeneous clients and achieve fast convergence of the global model. It's difficult to design the optimal dynamic learning rate for each client as the local information of other clients is unknown, especially during the local training epochs without communications between local clients and the central server. To enable a decentralized learning rate design for each client, we first introduce mean-field schemes to estimate the terms related to other clients' local model parameters. Then the decentralized adaptive learning rate for each client is obtained in closed form by constructing the Hamilton equation. Moreover, we prove that there exist fixed point solutions for the mean-field estimators, and an algorithm is proposed to obtain them. Finally, extensive experimental results on real datasets show that our algorithm can effectively eliminate the deviation between local model parameters compared to other recent FL algorithms.

翻译：联邦学习（Federated Learning, FL）近年来作为一种流行框架兴起，允许资源受限的离散客户端在中央服务器协调下协同学习全局模型，同时将隐私敏感数据存储在本地。然而，由于异构客户端的设备差异和数据分布差异，本地模型之间会产生参数偏差，导致收敛速度缓慢并降低全局模型的准确性。当前FL算法普遍采用静态客户端学习策略，无法适应不同客户端的动态训练参数。本文通过考虑不同本地模型参数之间的偏差，基于熵理论为每个客户端提出一种自适应学习率方案，以缓解异构客户端之间的偏差并实现全局模型的快速收敛。为每个客户端设计最优动态学习率具有挑战性，因为其他客户端的本地信息未知，尤其是在本地客户端与中央服务器之间无通信的本地训练轮次中。为了实现去中心化的学习率设计，我们首先引入平均场方案来估计与其他客户端本地模型参数相关的项。随后，通过构建哈密顿方程，以闭式形式获得每个客户端的去中心化自适应学习率。此外，我们证明了平均场估计器存在不动点解，并提出一种算法来获取这些解。最后，在真实数据集上的广泛实验结果表明，与近期其他FL算法相比，我们的算法能有效消除本地模型参数之间的偏差。