With the rapid proliferation of smart mobile devices, federated learning (FL) has been widely considered for application in wireless networks for distributed model training. However, data heterogeneity, e.g., non-independently identically distributions and different sizes of training data among clients, poses major challenges to wireless FL. Limited communication resources complicate the implementation of fair scheduling which is required for training on heterogeneous data, and further deteriorate the overall performance. To address this issue, this paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. Specifically, we first develop a closed-form expression for an upper bound on the FL loss function, with a particular emphasis on data heterogeneity described by a dataset size vector and a data divergence vector. Then we formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE). Next, via the Lyapunov drift technique, we transform the CRE optimization problem into a series of tractable problems. Extensive experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.
翻译:随着智能移动设备的快速普及,联邦学习(FL)被广泛考虑应用于无线网络中的分布式模型训练。然而,数据异构性(例如客户端间训练数据的非独立同分布特性及不同数据规模)给无线联邦学习带来了重大挑战。有限的通信资源使得为处理异构数据所需的公平调度难以实现,并进一步恶化整体性能。为解决这一问题,本文聚焦于结合数据异构性与无线资源分配的无线联邦学习性能分析与优化。具体而言,我们首先推导了联邦学习损失函数上界的闭式表达式,其中特别强调了由数据集大小向量和数据散度向量描述的数据异构性。随后,在长期能耗与延迟约束下,我们构建了损失函数最小化问题,并联合优化客户端调度、资源分配及本地训练轮次数(CRE)。接着,通过Lyapunov漂移技术,我们将CRE优化问题转化为一系列可解的子问题。在真实数据集上的大量实验表明,所提算法在学习精度与能耗方面均优于其他基准方案。