Federated learning (FL) is a popular machine learning technique that enables multiple users to collaboratively train a model while maintaining the user data privacy. A significant challenge in FL is the communication bottleneck in the upload direction, and thus the corresponding energy consumption of the devices, attributed to the increasing size of the model/gradient. In this paper, we address this issue by proposing a zero-order (ZO) optimization method that requires the upload of a quantized single scalar per iteration by each device instead of the whole gradient vector. We prove its theoretical convergence and find an upper bound on its convergence rate in the non-convex setting, and we discuss its implementation in practical scenarios. Our FL method and the corresponding convergence analysis take into account the impact of quantization and packet dropping due to wireless errors. We show also the superiority of our method, in terms of communication overhead and energy consumption, as compared to standard gradient-based FL methods.

翻译：联邦学习（FL）是一种流行的机器学习技术，它允许多个用户在保持用户数据隐私的同时协作训练模型。FL中的一个重大挑战是上传方向的通信瓶颈，以及由此导致的设备能耗增加，这归因于模型/梯度规模的不断增大。在本文中，我们通过提出一种零阶（ZO）优化方法来解决此问题，该方法要求每个设备在每次迭代时上传一个量化后的单一标量，而非整个梯度向量。我们证明了其理论收敛性，并在非凸设定下找到了其收敛速率的上界，同时讨论了其在实践场景中的实现。我们的FL方法及相应的收敛性分析考虑了量化及无线错误导致的数据包丢失的影响。我们还展示了与标准的基于梯度的FL方法相比，我们的方法在通信开销和能耗方面的优越性。