The optimal implementation of federated learning (FL) in practical edge computing systems has been an outstanding problem. In this paper, we propose an optimization-based quantized FL algorithm, which can appropriately fit a general edge computing system with uniform or nonuniform computing and communication resources at the workers. Specifically, we first present a new random quantization scheme and analyze its properties. Then, we propose a general quantized FL algorithm, namely GQFedWAvg. Specifically, GQFedWAvg applies the proposed quantization scheme to quantize wisely chosen model update-related vectors and adopts a generalized mini-batch stochastic gradient descent (SGD) method with the weighted average local model updates in global model aggregation. Besides, GQFedWAvg has several adjustable algorithm parameters to flexibly adapt to the computing and communication resources at the server and workers. We also analyze the convergence of GQFedWAvg. Next, we optimize the algorithm parameters of GQFedWAvg to minimize the convergence error under the time and energy constraints. We successfully tackle the challenging non-convex problem using general inner approximation (GIA) and multiple delicate tricks. Finally, we interpret GQFedWAvg's function principle and show its considerable gains over existing FL algorithms using numerical results.
翻译:联邦学习(FL)在实用边缘计算系统中的最优实现一直是一个悬而未决的问题。本文提出了一种基于优化的量化联邦学习算法,该算法能够适配工作节点具备均匀或非均匀计算与通信资源的通用边缘计算系统。具体而言,我们首先提出一种新的随机量化方案并分析其性质,随后设计通用量化联邦学习算法GQFedWAvg。该算法采用所提量化方案对明智选取的模型更新相关向量进行量化,并在全局模型聚合中引入采用加权平均本地模型更新的广义小批量随机梯度下降(SGD)方法。此外,GQFedWAvg包含多个可调算法参数,能灵活适配服务器与工作节点的计算及通信资源。我们分析了GQFedWAvg的收敛性,进而优化其算法参数以在时间与能量约束下最小化收敛误差,通过广义内近似(GIA)方法与多项精细技巧成功解决了这一非凸难题。最后,我们阐释了GQFedWAvg的工作原理,并利用数值结果展示了其相较于现有联邦学习算法的显著优势。