Mixture-of-Experts (MoE) architectures have become essential for scaling large language models, driving the development of specialized device-initiated communication libraries such as DeepEP, Hybrid-EP, and others. These libraries demonstrate the performance benefits of GPU-initiated RDMA for MoE dispatch and combine operations. This paper presents NCCL EP (Expert Parallelism), a ground-up MoE communication library built entirely on NCCL's Device API. NCCL EP provides unified ncclEpDispatch and ncclEpCombine primitives with both C and Python interfaces, supporting Low-Latency (LL) mode for inference decoding and High-Throughput (HT) mode for training and inference prefill. LL targets small batch sizes (1-128 tokens) using direct all-to-all RDMA+NVLink mesh connectivity with double-buffered communication for overlapping dispatch and combine phases. HT targets large batches (4096+ tokens) using hierarchical communication that aggregates tokens within NVLink domains before inter-node RDMA transmission. Both modes leverage Device API for both intra- and inter-node communications, taking advantage of its topology awareness and optimized GPU-initiated implementation. We evaluate NCCL EP on an H100-based cluster across multi-node configurations, demonstrating competitive LL kernel performance and presenting end-to-end results with vLLM integration. By building MoE communication natively within NCCL, NCCL EP provides a supported path for expert parallelism on current and emerging NVIDIA platforms.

翻译：混合专家（MoE）架构已成为扩展大语言模型的关键技术，推动了DeepEP、Hybrid-EP等专用设备端发起通信库的发展。这些库展示了GPU端发起RDMA在MoE分发与聚合操作中的性能优势。本文提出NCCL EP（专家并行性），这是一个完全基于NCCL设备API从头构建的MoE通信库。NCCL EP通过C与Python接口提供统一的ncclEpDispatch和ncclEpCombine原语，支持面向推理解码的低延迟（LL）模式以及面向训练与推理预填充的高吞吐（HT）模式。LL模式针对小批量（1-128个令牌）场景，采用直接全对全RDMA+NVLink网状连接，并通过双缓冲通信实现分发与聚合阶段的重叠。HT模式针对大批量（4096+以上令牌）场景，采用分层通信策略，先在NVLink域内聚合令牌，再进行节点间RDMA传输。两种模式均利用设备API实现节点内与节点间通信，充分发挥其拓扑感知与GPU端发起优化的优势。我们在基于H100的多节点集群上评估NCCL EP，展示了具有竞争力的LL内核性能，并通过vLLM集成呈现端到端实验结果。通过将MoE通信原生构建于NCCL内部，NCCL EP为当前及未来NVIDIA平台上的专家并行性提供了可持续的技术支持路径。