Federated Learning (FL) has recently emerged as a popular framework, which allows resource-constrained discrete clients to cooperatively learn the global model under the orchestration of a central server while storing privacy-sensitive data locally. However, due to the difference in equipment and data divergence of heterogeneous clients, there will be parameter deviation between local models, resulting in a slow convergence rate and a reduction of the accuracy of the global model. The current FL algorithms use the static client learning strategy pervasively and can not adapt to the dynamic training parameters of different clients. In this paper, by considering the deviation between different local model parameters, we propose an adaptive learning rate scheme for each client based on entropy theory to alleviate the deviation between heterogeneous clients and achieve fast convergence of the global model. It's difficult to design the optimal dynamic learning rate for each client as the local information of other clients is unknown, especially during the local training epochs without communications between local clients and the central server. To enable a decentralized learning rate design for each client, we first introduce mean-field schemes to estimate the terms related to other clients' local model parameters. Then the decentralized adaptive learning rate for each client is obtained in closed form by constructing the Hamilton equation. Moreover, we prove that there exist fixed point solutions for the mean-field estimators, and an algorithm is proposed to obtain them. Finally, extensive experimental results on real datasets show that our algorithm can effectively eliminate the deviation between local model parameters compared to other recent FL algorithms.

翻译：联邦学习（FL）近期成为一种流行框架，它允许资源受限的离散客户端在中央服务器的协调下协作学习全局模型，同时将隐私敏感数据存储在本地。然而，由于异构客户端设备差异和数据分布偏差，本地模型之间会出现参数偏离，导致收敛速度缓慢并降低全局模型的精度。当前FL算法普遍采用静态客户端学习策略，无法适应不同客户端的动态训练参数。本文通过考虑不同本地模型参数之间的偏差，提出了一种基于熵理论的各客户端自适应学习率方案，以缓解异构客户端之间的偏差并实现全局模型的快速收敛。由于其他客户端的本地信息未知，尤其是在本地客户端与中央服务器无通信的本地训练周期内，为每个客户端设计最优动态学习率十分困难。为了实现每个客户端的分散式学习率设计，我们首先引入平均场方案来估计与其他客户端本地模型参数相关的项。然后通过构建哈密顿方程，以闭式形式获得每个客户端的分散式自适应学习率。此外，我们证明了平均场估计器存在不动点解，并提出相应算法来获取这些解。最后，在真实数据集上的大量实验结果表明，与近期其他FL算法相比，我们的算法能有效消除本地模型参数之间的偏差。