Federated learning (FL) enables distributed clients to collaboratively train a machine learning model without sharing raw data with each other. However, it suffers the leakage of private information from uploading models. In addition, as the model size grows, the training latency increases due to limited transmission bandwidth and the model performance degrades while using differential privacy (DP) protection. In this paper, we propose a gradient sparsification empowered FL framework over wireless channels, in order to improve training efficiency without sacrificing convergence performance. Specifically, we first design a random sparsification algorithm to retain a fraction of the gradient elements in each client's local training, thereby mitigating the performance degradation induced by DP and and reducing the number of transmission parameters over wireless channels. Then, we analyze the convergence bound of the proposed algorithm, by modeling a non-convex FL problem. Next, we formulate a time-sequential stochastic optimization problem for minimizing the developed convergence bound, under the constraints of transmit power, the average transmitting delay, as well as the client's DP requirement. Utilizing the Lyapunov drift-plus-penalty framework, we develop an analytical solution to the optimization problem. Extensive experiments have been implemented on three real life datasets to demonstrate the effectiveness of our proposed algorithm. We show that our proposed algorithms can fully exploit the interworking between communication and computation to outperform the baselines, i.e., random scheduling, round robin and delay-minimization algorithms.

翻译：联邦学习（FL）使分布式客户端能够在不共享原始数据的情况下协同训练机器学习模型。然而，其存在因上传模型导致的隐私信息泄露风险。此外，随着模型规模增大，有限传输带宽导致训练延迟增加，而采用差分隐私（DP）保护时模型性能会下降。本文提出一种基于梯度稀疏化的无线信道联邦学习框架，旨在不牺牲收敛性能的前提下提升训练效率。具体而言，我们首先设计随机稀疏化算法，在客户端本地训练中保留部分梯度元素，从而减轻DP导致的性能退化并减少无线信道传输参数数量。随后，通过建模非凸FL问题分析所提算法的收敛界。接着，在发射功率、平均传输时延及客户端DP需求的约束下，构建最小化收敛界的时间序列随机优化问题。利用Lyapunov漂移加罚框架，推导出优化问题的解析解。基于三个真实数据集的大量实验验证了所提算法的有效性。结果表明，我们的算法能充分利用通信与计算的协同作用，优于随机调度、轮询调度和延迟最小化等基线算法。