While data is distributed in multiple edge devices, Federated Learning (FL) is attracting more and more attention to collaboratively train a machine learning model without transferring raw data. FL generally exploits a parameter server and a large number of edge devices during the whole process of the model training, while several devices are selected in each round. However, straggler devices may slow down the training process or even make the system crash during training. Meanwhile, other idle edge devices remain unused. As the bandwidth between the devices and the server is relatively low, the communication of intermediate data becomes a bottleneck. In this paper, we propose Time-Efficient Asynchronous federated learning with Sparsification and Quantization, i.e., TEASQ-Fed. TEASQ-Fed can fully exploit edge devices to asynchronously participate in the training process by actively applying for tasks. We utilize control parameters to choose an appropriate number of parallel edge devices, which simultaneously execute the training tasks. In addition, we introduce a caching mechanism and weighted averaging with respect to model staleness to further improve the accuracy. Furthermore, we propose a sparsification and quantitation approach to compress the intermediate data to accelerate the training. The experimental results reveal that TEASQ-Fed improves the accuracy (up to 16.67% higher) while accelerating the convergence of model training (up to twice faster).

翻译：数据分布存储在多个边缘设备中，联邦学习因其无需传输原始数据即可协同训练机器学习模型而备受关注。联邦学习通常在整个模型训练过程中利用参数服务器和大量边缘设备，每轮训练会选取若干设备参与。然而，掉队设备可能拖慢训练进程，甚至在训练期间导致系统崩溃，同时其他空闲边缘设备却未被利用。由于设备与服务器之间的带宽相对较低，中间数据的通信成为瓶颈。本文提出一种基于稀疏化和量化的时间高效异步联邦学习算法（TEASQ-Fed），该算法通过设备主动申请任务的方式，充分利用边缘设备异步参与训练过程。我们采用控制参数选择适当数量的并行边缘设备同时执行训练任务，并引入缓存机制和基于模型陈旧度的加权平均方法以进一步提升精度。此外，我们提出一种稀疏化与量化方法来压缩中间数据以加速训练。实验结果表明，TEASQ-Fed 在加速模型训练收敛（最高提升2倍）的同时，提高了模型精度（最高提升16.67%）。