To meet strict Service-Level Objectives (SLOs),contemporary Large Language Models (LLMs) decouple the prefill and decoding stages and place them on separate GPUs to mitigate the distinct bottlenecks inherent to each phase. However, the heterogeneity of LLM workloads causes producerconsumer imbalance between the two instance types in such disaggregated architecture. To address this problem, we propose DOPD (Dynamic Optimal Prefill/Decoding), a dynamic LLM inference system that adjusts instance allocations to achieve an optimal prefill-to-decoding (P/D) ratio based on real-time load monitoring. Combined with an appropriate request-scheduling policy, DOPD effectively resolves imbalances between prefill and decoding instances and mitigates resource allocation mismatches due to mixed-length requests under high concurrency. Experimental evaluations show that, compared with vLLM and DistServe (representative aggregation-based and disaggregationbased approaches), DOPD improves overall system goodput by up to 1.5X, decreases P90 time-to-first-token (TTFT) by up to 67.5%, and decreases P90 time-per-output-token (TPOT) by up to 22.8%. Furthermore, our dynamic P/D adjustment technique performs proactive reconfiguration based on historical load, achieving over 99% SLOs attainment while using less additional resources.

翻译：为满足严格的服务级别目标（SLO），当代大型语言模型（LLMs）将预填充和解码阶段解耦，并将其置于独立的GPU上，以缓解各阶段固有的不同瓶颈。然而，LLM工作负载的异构性导致在此类解耦架构中，两种实例类型之间出现生产者-消费者失衡。为解决此问题，我们提出了DOPD（动态最优预填充/解码），这是一种动态LLM推理系统，可根据实时负载监控调整实例分配，以实现最优的预填充-解码（P/D）比率。结合适当的请求调度策略，DOPD有效解决了预填充与解码实例间的失衡，并缓解了高并发下混合长度请求导致的资源分配失配问题。实验评估表明，与vLLM和DistServe（分别代表基于聚合与基于解耦的代表性方法）相比，DOPD将系统整体有效吞吐量提升最高达1.5倍，将P90首词元时间（TTFT）降低最高达67.5%，并将P90单输出词元时间（TPOT）降低最高达22.8%。此外，我们的动态P/D调整技术基于历史负载进行主动重配置，在占用较少额外资源的同时，实现了超过99%的SLO达成率。