The optimal implementation of federated learning (FL) in practical edge computing systems has been an outstanding problem. In this paper, we propose an optimization-based quantized FL algorithm, which can appropriately fit a general edge computing system with uniform or nonuniform computing and communication resources at the workers. Specifically, we first present a new random quantization scheme and analyze its properties. Then, we propose a general quantized FL algorithm, namely GQFedWAvg. Specifically, GQFedWAvg applies the proposed quantization scheme to quantize wisely chosen model update-related vectors and adopts a generalized mini-batch stochastic gradient descent (SGD) method with the weighted average local model updates in global model aggregation. Besides, GQFedWAvg has several adjustable algorithm parameters to flexibly adapt to the computing and communication resources at the server and workers. We also analyze the convergence of GQFedWAvg. Next, we optimize the algorithm parameters of GQFedWAvg to minimize the convergence error under the time and energy constraints. We successfully tackle the challenging non-convex problem using general inner approximation (GIA) and multiple delicate tricks. Finally, we interpret GQFedWAvg's function principle and show its considerable gains over existing FL algorithms using numerical results.
翻译:联邦学习在实际边缘计算系统中的最优实现一直是一个悬而未决的问题。本文提出了一种基于优化的量化联邦学习算法,该算法能够很好地适配工作节点计算和通信资源均匀或非均匀的通用边缘计算系统。具体而言,我们首先提出了一种新的随机量化方案并分析了其性质。然后,我们提出了一种通用量化联邦学习算法,即GQFedWAvg。具体地,GQFedWAvg应用所提出的量化方案来量化经过明智选择的模型更新相关向量,并在全局模型聚合中采用带有加权平均局部模型更新的广义小批量随机梯度下降(SGD)方法。此外,GQFedWAvg具有多个可调节的算法参数,以灵活适应服务器和工作节点的计算与通信资源。我们还分析了GQFedWAvg的收敛性。接下来,我们优化GQFedWAvg的算法参数,以在时间和能量约束下最小化收敛误差。我们通过通用内近似(GIA)和多种精巧技巧成功解决了这一具有挑战性的非凸问题。最后,我们阐释了GQFedWAvg的工作原理,并通过数值结果展示了其相对于现有联邦学习算法的显著优势。