Prefill-decode (PD) disaggregation has become the standard architecture for large-scale LLM serving, but in practice its deployment boundary is still determined by KVCache transfer. In conventional dense-attention models, prefill generates huge KVCache traffics that keep prefill and decode tightly coupled within a single high-bandwidth network domain, limiting heterogeneous deployment and resource elasticity. Recent hybrid-attention architectures substantially reduce KVCache size, making cross-cluster KVCache transport increasingly plausible. However, smaller KVCache alone does not make heterogeneous cross-datacenter PD serving practical: real workloads remain bursty, request lengths are highly skewed, prefix caches are unevenly distributed, and inter-cluster bandwidth fluctuates. A naive design that fully externalizes prefill can therefore still suffer from congestion, unstable queueing, and poor utilization. We present Prefill-as-a-Service (PrfaaS), a cross-datacenter serving architecture that selectively offloads long-context prefill to standalone, compute-dense prefill clusters and transfers the resulting KVCache over commodity Ethernet to local PD clusters for decode. Rather than treating reduced KVCache as sufficient, PrfaaS combines model-side KV efficiency with system-side selective offloading, bandwidth-aware scheduling, and cache-aware request placement. This design removes the requirement that heterogeneous accelerators share the same low-latency RDMA fabric, enabling independent scaling of prefill and decode capacity across loosely coupled clusters. In a case study using an internal 1T-parameter hybrid model, a PrfaaS-augmented heterogeneous deployment achieves 54% higher serving throughput and 64% lower P90 TTFT than a homogeneous PD baseline, with approximately 15% throughput gain at equal cost, while consuming only modest cross-datacenter bandwidth.

翻译：预填-解码（PD）分离已成为大规模LLM服务的标准架构，但在实际应用中，其部署边界仍受限于KVCache传输。在传统密集注意力模型中，预填过程产生巨大的KVCache通信量，导致预填与解码紧密耦合在单一高带宽网络域内，限制了异构部署和资源弹性。近年来的混合注意力架构显著减小了KVCache规模，使得跨集群KVCache传输逐渐成为可能。然而，仅凭更小的KVCache无法实现异构跨数据中心PD服务：真实工作负载仍具有突发性，请求长度高度偏斜，前缀缓存分布不均，且集群间带宽波动剧烈。因此，若采用完全外化预填的朴素设计，仍可能遭遇拥塞、队列不稳定及利用率低下等问题。我们提出“即预填即服务”（PrfaaS）——一种跨数据中心服务架构，该架构选择性地将长上下文预填任务卸载至独立的、计算密集的预填集群，并通过商用以太网将生成的KVCache传输至本地PD集群进行解码。PrfaaS并未将减小KVCache视为充分条件，而是结合模型侧的KV效率与系统侧的选择性卸载、带宽感知调度及缓存感知请求放置。该设计消除了异构加速器共享同一低延迟RDMA网络的需求，支持在松散耦合集群间独立扩展预填与解码容量。基于内部1T参数混合模型的案例研究显示，与同构PD基线相比，采用PrfaaS增强的异构部署实现了54%的服务吞吐量提升和64%的P90 TTFT降低，在同等成本下吞吐量增益约15%，且仅消耗适度的跨数据中心带宽。