The increasingly deeper neural networks hinder the democratization of privacy-enhancing distributed learning, such as federated learning (FL), to resource-constrained devices. To overcome this challenge, in this paper, we advocate the integration of edge computing paradigm and parallel split learning (PSL), allowing multiple client devices to offload substantial training workloads to an edge server via layer-wise model split. By observing that existing PSL schemes incur excessive training latency and large volume of data transmissions, we propose an innovative PSL framework, namely, efficient parallel split learning (EPSL), to accelerate model training. To be specific, EPSL parallelizes client-side model training and reduces the dimension of local gradients for back propagation (BP) via last-layer gradient aggregation, leading to a significant reduction in server-side training and communication latency. Moreover, by considering the heterogeneous channel conditions and computing capabilities at client devices, we jointly optimize subchannel allocation, power control, and cut layer selection to minimize the per-round latency. Simulation results show that the proposed EPSL framework significantly decreases the training latency needed to achieve a target accuracy compared with the state-of-the-art benchmarks, and the tailored resource management and layer split strategy can considerably reduce latency than the counterpart without optimization.

翻译：日益加深的神经网络阻碍了隐私增强型分布式学习（如联邦学习）在资源受限设备上的普及。为克服这一挑战，本文倡导将边缘计算范式与并行分割学习相结合，通过按层模型拆分，允许多个客户端设备将大量训练负载卸载至边缘服务器。针对现有并行分割学习方案存在训练延迟过高和数据传输量过大的问题，我们提出了一种创新的并行分割学习框架——高效并行分割学习，用于加速模型训练。具体而言，EPSL通过并行化客户端模型训练，并利用最后一层梯度聚合降低反向传播所需的本地梯度维度，从而显著减少服务器端训练和通信延迟。此外，考虑到客户端设备异构的信道条件和计算能力，我们联合优化子信道分配、功率控制和切层选择以最小化每轮训练延迟。仿真结果表明，与现有最优基准方案相比，所提EPSL框架在达到目标精度时显著降低了训练延迟；同时，定制化的资源管理与切层策略相比未经优化的方案能大幅减少延迟。