In this paper, a new communication-efficient federated learning (FL) framework is proposed, inspired by vector quantized compressed sensing. The basic strategy of the proposed framework is to compress the local model update at each device by applying dimensionality reduction followed by vector quantization. Subsequently, the global model update is reconstructed at a parameter server by applying a sparse signal recovery algorithm to the aggregation of the compressed local model updates. By harnessing the benefits of both dimensionality reduction and vector quantization, the proposed framework effectively reduces the communication overhead of local update transmissions. Both the design of the vector quantizer and the key parameters for the compression are optimized so as to minimize the reconstruction error of the global model update under the constraint of wireless link capacity. By considering the reconstruction error, the convergence rate of the proposed framework is also analyzed for a non-convex loss function. Simulation results on the MNIST and FEMNIST datasets demonstrate that the proposed framework provides more than a 2.4% increase in classification accuracy compared to state-of-the-art FL frameworks when the communication overhead of the local model update transmission is 0.1 bit per local model entry.

翻译：本文提出了一种受向量量化压缩感知启发的通信高效联邦学习（FL）框架。该框架的基本策略是通过降维结合向量量化来压缩各设备上的局部模型更新。随后，参数服务器通过对压缩后的局部模型更新进行聚合，并应用稀疏信号恢复算法重建全局模型更新。该框架利用降维和向量量化的双重优势，有效降低了局部更新传输的通信开销。通过优化向量量化器的设计以及压缩的关键参数，在无线链路容量约束下最小化全局模型更新的重建误差。在考虑重建误差的基础上，本文还分析了该框架在非凸损失函数下的收敛速度。在MNIST和FEMNIST数据集上的仿真结果表明，当局部模型更新传输的通信开销为每局部模型参数0.1比特时，与最先进的FL框架相比，所提框架的分类准确率提升超过2.4%。