The increasingly deeper neural networks hinder the democratization of privacy-enhancing distributed learning, such as federated learning (FL), to resource-constrained devices. To overcome this challenge, in this paper, we advocate the integration of edge computing paradigm and parallel split learning (PSL), allowing multiple client devices to offload substantial training workloads to an edge server via layer-wise model split. By observing that existing PSL schemes incur excessive training latency and large volume of data transmissions, we propose an innovative PSL framework, namely, efficient parallel split learning (EPSL), to accelerate model training. To be specific, EPSL parallelizes client-side model training and reduces the dimension of local gradients for back propagation (BP) via last-layer gradient aggregation, leading to a significant reduction in server-side training and communication latency. Moreover, by considering the heterogeneous channel conditions and computing capabilities at client devices, we jointly optimize subchannel allocation, power control, and cut layer selection to minimize the per-round latency. Simulation results show that the proposed EPSL framework significantly decreases the training latency needed to achieve a target accuracy compared with the state-of-the-art benchmarks, and the tailored resource management and layer split strategy can considerably reduce latency than the counterpart without optimization.

翻译：日益深化的神经网络阻碍了隐私增强型分布式学习（如联邦学习）在资源受限设备上的普及。为克服这一挑战，本文倡导将边缘计算范式与并行分割学习（PSL）相结合，允许客户端设备通过层级模型分割将繁重的训练负载卸载至边缘服务器。针对现有PSL方案存在训练延迟过高与数据传输量过大的问题，我们提出一种创新的PSL框架——高效并行分割学习（EPSL）以加速模型训练。具体而言，EPSL通过并行化客户端模型训练，并采用末层梯度聚合技术降低反向传播（BP）所需局部梯度的维度，从而显著减少服务器端训练与通信延迟。此外，考虑到客户端设备异构的信道条件与计算能力，我们联合优化子信道分配、功率控制及分割层选择，以最小化每轮训练延迟。仿真结果表明，与现有基准方案相比，所提EPSL框架在达到目标精度时显著降低了训练延迟，且定制化的资源管理与层级分割策略相较于未优化方案可大幅缩短延迟。