Modern data parallel (DP) training favors collective communication over parameter servers (PS) for its simplicity and efficiency under balanced workloads. However, the balanced workload assumption no longer holds in large language model (LLM) post-training due to the high variance in sequence lengths. Under imbalanced workloads, collective communication creates synchronization barriers, leading to under-utilization of devices with smaller workloads. This change in training dynamics calls for a revisit of the PS paradigm for its robustness to such imbalance. We propose \textbf{On-Demand Communication (ODC)}, which adapts PS into Fully Sharded Data Parallel (FSDP) by replacing collective all-gather and reduce-scatter with direct point-to-point communication. Compared to FSDP, ODC reduces the synchronization barrier from once per layer to once per minibatch and decouples the workload on each device so that faster workers are not stalled. It also enables simpler and more effective load balancing at the minibatch level. Across diverse LLM post-training tasks, ODC consistently improves device utilization and training throughput, achieving up to a 36\% speedup over standard FSDP. These results demonstrate that ODC is a superior fit for the prevalent imbalanced workloads in LLM post-training. Our implementation of ODC and integration with FSDP is open-sourced at https://github.com/sail-sg/odc.

翻译：现代数据并行训练因其在均衡负载下的简洁性和高效性，通常采用集合通信而非参数服务器。然而，在大语言模型后训练中，由于序列长度的高度差异性，均衡负载的假设不再成立。在负载不均衡的情况下，集合通信会创建同步屏障，导致负载较小的设备利用率不足。这种训练动态的变化促使我们重新审视参数服务器范式，因其对此类不均衡情况具有鲁棒性。我们提出 **按需通信**，通过将集合式全收集与规约分散操作替换为直接的点对点通信，将参数服务器范式适配到全分片数据并行中。与全分片数据并行相比，按需通信将同步屏障从每层一次减少为每个小批次一次，并解耦了各设备上的工作负载，使得更快的计算节点不会被阻塞。它还支持在小批次级别实现更简单且更有效的负载均衡。在多样的大语言模型后训练任务中，按需通信持续提升了设备利用率和训练吞吐量，相比标准全分片数据并行实现了高达36%的加速。这些结果表明，按需通信非常适合大语言模型后训练中普遍存在的不均衡工作负载。我们的按需通信实现及其与全分片数据并行的集成已在 https://github.com/sail-sg/odc 开源。