Scaled-out MoE LLMs and scaled-up SuperPods create new systems challenges for production Model-as-a-Service (MaaS), requiring disaggregation, low-latency communication, and decentralized serving. This report presents xDeepServe, the production serving system behind Huawei Cloud's MaaS offering on CloudMatrix384, a 48-server SuperPod with 384 Ascend 910C chips connected by a high-bandwidth UB fabric and global shared memory. It serves models including DeepSeek, Kimi, GLM, Qwen, and MiniMax, among others. xDeepServe is built around Transformerless, a disaggregated execution architecture that decomposes transformer inference into modular units -- attention, feedforward, and MoE -- and supports disaggregated Prefill-Decode and MoE-Attention deployments. To enable disaggregation, we develop XCCL, a memory-semantic communication layer providing microsecond-level point-to-point and scalable all-to-all primitives, and we extend FlowServe with decentralized DP groups and techniques to mitigate stragglers and synchronization variance. In a peak decoding configuration, xDeepServe reaches 2400 tokens/s per Ascend 910C chip at ~50ms time-per-output-token (TPOT).

翻译：规模化扩展的MoE大语言模型与超级集群的规模提升为生产级模型即服务（MaaS）带来了新的系统挑战，需要实现解耦架构、低延迟通信与去中心化服务。本报告介绍了xDeepServe——支撑华为云在CloudMatrix384超级集群上提供MaaS服务的生产级服务系统。该超级集群由48台服务器组成，搭载384颗Ascend 910C芯片，通过高带宽UB互连架构与全局共享内存连接。该系统服务于包括DeepSeek、Kimi、GLM、Qwen及MiniMax在内的多种模型。xDeepServe围绕Transformerless构建，这是一种解耦式执行架构，将Transformer推理分解为注意力、前馈网络和MoE等模块化单元，并支持解耦的预填充-解码与MoE-注意力部署。为实现解耦，我们开发了XCCL内存语义通信层，提供微秒级点对点通信及可扩展的全对全通信原语；同时扩展FlowServe系统，引入去中心化数据并行组及消除拖尾效应与同步波动的技术。在峰值解码配置下，xDeepServe在每颗Ascend 910C芯片上实现约2400 tokens/s的吞吐量，单token输出时间（TPOT）约为50毫秒。