The traditional framework of federated learning (FL) requires each client to re-train their models in every iteration, making it infeasible for resource-constrained mobile devices to train deep-learning (DL) models. Split learning (SL) provides an alternative by using a centralized server to offload the computation of activations and gradients for a subset of the model but suffers from problems of slow convergence and lower accuracy. In this paper, we implement PFSL, a new framework of distributed split learning where a large number of thin clients perform transfer learning in parallel, starting with a pre-trained DL model without sharing their data or labels with a central server. We implement a lightweight step of personalization of client models to provide high performance for their respective data distributions. Furthermore, we evaluate performance fairness amongst clients under a work fairness constraint for various scenarios of non-i.i.d. data distributions and unequal sample sizes. Our accuracy far exceeds that of current SL algorithms and is very close to that of centralized learning on several real-life benchmarks. It has a very low computation cost compared to FL variants and promises to deliver the full benefits of DL to extremely thin, resource-constrained clients.

翻译：联邦学习（FL）的传统框架要求每个客户端在每次迭代中重新训练其模型，这使得资源受限的移动设备无法训练深度学习（DL）模型。分割学习（SL）提供了一种替代方案，通过使用集中式服务器来卸载模型子集的激活和梯度计算，但存在收敛速度慢和精度较低的问题。在本文中，我们实现了PFSL，这是一种新型分布式分割学习框架，其中大量瘦客户端并行执行迁移学习，从预训练的DL模型开始，无需与中央服务器共享其数据或标签。我们实现了一个轻量级的客户端模型个性化步骤，以针对各自的数据分布提供高性能。此外，我们评估了在非独立同分布（non-i.i.d.）数据分布和不等样本量的各种场景下，客户端在工作公平约束下的性能公平性。我们的精度远远超过当前的SL算法，并且在多个实际基准测试中接近集中式学习的精度。与FL变体相比，它具有极低的计算成本，有望将DL的全部优势带给极其瘦小且资源受限的客户端。