In this study the problem of Federated Learning (FL) is explored under a new perspective by utilizing the Deep Equilibrium (DEQ) models instead of conventional deep learning networks. We claim that incorporating DEQ models into the federated learning framework naturally addresses several open problems in FL, such as the communication overhead due to the sharing large models and the ability to incorporate heterogeneous edge devices with significantly different computation capabilities. Additionally, a weighted average fusion rule is proposed at the server-side of the FL framework to account for the different qualities of models from heterogeneous edge devices. To the best of our knowledge, this study is the first to establish a connection between DEQ models and federated learning, contributing to the development of an efficient and effective FL framework. Finally, promising initial experimental results are presented, demonstrating the potential of this approach in addressing challenges of FL.

翻译：本研究从新视角探讨联邦学习问题，采用深度均衡模型替代传统深度学习网络。我们提出，将深度均衡模型融入联邦学习框架可自然解决联邦学习中的若干开放问题，如共享大模型带来的通信开销以及整合计算能力差异显著的异构边缘设备的能力。此外，在联邦学习框架的服务端提出一种加权平均融合规则，以考虑来自异构边缘设备的不同质量模型。据我们所知，本研究首次建立了深度均衡模型与联邦学习之间的联系，为开发高效且有效的联邦学习框架做出了贡献。最后，本文展示了初步实验成果，证明了该方法在应对联邦学习挑战方面的潜力。