Cross-device federated learning (FL) is a growing machine learning setting whereby multiple edge devices collaborate to train a model without disclosing their raw data. With the great number of mobile devices participating in more FL applications via the wireless environment, the practical implementation of these applications will be hindered due to the limited uplink capacity of devices, causing critical bottlenecks. In this work, we propose a novel doubly communication-efficient zero-order (ZO) method with a one-point gradient estimator that replaces communicating long vectors with scalar values and that harnesses the nature of the wireless communication channel, overcoming the need to know the channel state coefficient. It is the first method that includes the wireless channel in the learning algorithm itself instead of wasting resources to analyze it and remove its impact. We then offer a thorough analysis of the proposed zero-order federated learning (ZOFL) framework and prove that our method converges \textit{almost surely}, which is a novel result in nonconvex ZO optimization. We further prove a convergence rate of $O(\frac{1}{\sqrt[3]{K}})$ in the nonconvex setting. We finally demonstrate the potential of our algorithm with experimental results.

翻译：跨设备联邦学习（FL）是一种新兴的机器学习范式，允许多个边缘设备在不共享原始数据的情况下协作训练模型。随着大量移动设备通过无线环境参与更多联邦学习应用，设备有限的上行链路容量将阻碍这些应用的实际部署，造成关键瓶颈。本研究提出一种新颖的双重通信高效零阶方法，采用单点梯度估计器，将长向量通信替换为标量值传输，并利用无线通信信道的固有特性，从而无需获知信道状态系数。这是首个将无线信道直接纳入学习算法本身的方法，而非浪费资源分析并消除其影响。随后，我们对所提出的零阶联邦学习框架进行了深入分析，证明了该方法在非凸零阶优化中具有\textit{几乎必然}收敛性——这是非凸零阶优化领域的新颖结论。我们进一步证明了在非凸设定下该方法具有$O(\frac{1}{\sqrt[3]{K}})$的收敛速率。最后通过实验结果展示了该算法的潜力。