The increasingly deeper neural networks hinder the democratization of privacy-enhancing distributed learning, such as federated learning (FL), to resource-constrained devices. To overcome this challenge, in this paper, we advocate the integration of edge computing paradigm and parallel split learning (PSL), allowing multiple client devices to offload substantial training workloads to an edge server via layer-wise model split. By observing that existing PSL schemes incur excessive training latency and large volume of data transmissions, we propose an innovative PSL framework, namely, efficient parallel split learning (EPSL), to accelerate model training. To be specific, EPSL parallelizes client-side model training and reduces the dimension of local gradients for back propagation (BP) via last-layer gradient aggregation, leading to a significant reduction in server-side training and communication latency. Moreover, by considering the heterogeneous channel conditions and computing capabilities at client devices, we jointly optimize subchannel allocation, power control, and cut layer selection to minimize the per-round latency. Simulation results show that the proposed EPSL framework significantly decreases the training latency needed to achieve a target accuracy compared with the state-of-the-art benchmarks, and the tailored resource management and layer split strategy can considerably reduce latency than the counterpart without optimization.

翻译：日益加深的神经网络阻碍了隐私增强型分布式学习（如联邦学习，FL）在资源受限设备上的普及。为应对这一挑战，本文倡导将边缘计算范式与并行分割学习（PSL）相结合，允许多个客户端设备通过逐层模型分割将繁重的训练工作负载卸载到边缘服务器。针对现有PSL方案存在训练延迟过高和数据传输量过大的问题，我们提出了一种创新的PSL框架——高效并行分割学习（EPSL），以加速模型训练。具体而言，EPSL并行化客户端模型训练，并通过末层梯度聚合降低反向传播（BP）中本地梯度的维度，从而显著减少服务器端训练和通信延迟。此外，考虑客户端设备的异构信道条件和计算能力，我们联合优化子信道分配、功率控制和切割层选择，以最小化每轮训练延迟。仿真结果表明，相比最先进的基准方案，所提出的EPSL框架在达到目标精度所需训练延迟上显著降低；且针对性的资源管理与层分割策略相较于未经优化的方案可大幅减少延迟。