Parallel split learning (PSL) suffers from two intertwined issues: the effective batch size grows with the number of clients, and data that is not identically and independently distributed (non-IID) skews global batches. We present parallel split learning with global sampling (GPSL), a server-driven scheme that fixes the global batch size while computing per-client batch-size schedules using pooled-level proportions. The actual samples are drawn locally without replacement by each selected client. This eliminates per-class rounding, decouples the effective batch from the client count, and makes each global batch distributionally equivalent to centralized uniform sampling without replacement. Consequently, we obtain finite-population deviation guarantees via Serfling's inequality, yielding a zero rounding bias compared to local sampling schemes. GPSL is a drop-in replacement for PSL with negligible overhead and scales to large client populations. In extensive experiments on CIFAR-10/100 and ResNet-18/34 under non-IID splits, GPSL stabilizes optimization and achieves centralized-like accuracy, while fixed local batching trails by up to 60%. Furthermore, GPSL shortens training time by avoiding inflation of training steps induced by data-depletion. These findings suggest GPSL is a promising and scalable approach for learning in resource-constrained environments.

翻译：并行分割学习（PSL）面临两个相互交织的问题：有效批大小随客户端数量增长，而非独立同分布（non-IID）数据会扭曲全局批次。我们提出了一种基于全局采样的并行分割学习方法（GPSL），这是一种服务器驱动的方案，通过使用池化层级比例计算各客户端批大小调度，同时固定全局批大小。实际样本由每个被选中的客户端在本地无放回地抽取。该方法消除了按类舍入误差，将有效批次与客户端数量解耦，并使每个全局批次在分布上等价于集中式无放回均匀采样。因此，我们通过Serfling不等式获得了有限总体偏差保证，与本地采样方案相比实现了零舍入偏差。GPSL可作为PSL的直接替代方案，其开销可忽略不计，并能扩展至大规模客户端群体。在CIFAR-10/100数据集和ResNet-18/34模型上进行的非独立同分布分割大量实验中，GPSL稳定了优化过程并实现了接近集中式训练的精度，而固定本地批处理方法性能落后高达60%。此外，GPSL通过避免数据耗尽导致的训练步骤膨胀，缩短了训练时间。这些结果表明，GPSL是一种适用于资源受限环境下学习的高效可扩展方法。