Prefill/decode disaggregation is increasingly adopted in LLM serving to improve the latency-throughput tradeoff and meet strict TTFT and TPOT SLOs. However, LLM inference remains energy-hungry: autoscaling alone is too coarse-grained to track fast workload fluctuations, and applying fine-grained DVFS under disaggregation is complicated by phase-asymmetric dynamics and coupling between provisioning and frequency control. We present DualScale, a two-tier energy optimization framework for disaggregated LLM serving. DualScale jointly optimizes placement and DVFS across prefill and decode using predictive latency and power models. At coarse timescales, DualScale computes phase-aware placement and baseline frequencies that minimize energy while satisfying SLO constraints. At fine timescales, DualScale dynamically adapts GPU frequency per iteration using stage-specific control: model predictive control (MPC) for prefill to account for queue evolution and future TTFT impact, and lightweight slack-aware adaptation for decode to exploit its smoother, memory-bound dynamics. This hierarchical design enables coordinated control across timescales while preserving strict serving SLOs. Evaluation on a 16x H100 cluster serving Llama 3.3 70B with production-style traces shows that DualScale meets TTFT/TPOT SLOs while reducing energy by up to 39% in prefill and 48% in decode relative to DistServe.

翻译：预填充/解码分离技术被越来越多地应用于大语言模型（LLM）服务中，以改善延迟-吞吐量权衡，并满足严格的TTFT和TPOT服务等级目标。然而，LLM推理仍然耗能严重：仅靠自动扩缩容过于粗粒度，难以追踪工作负载的快速波动；而在分离式架构下应用细粒度DVFS，则因相位非对称动态特性以及资源调配与频率控制之间的耦合而变得复杂。本文提出双尺度（DualScale）——一种面向分离式LLM服务的双层能效优化框架。DualScale利用预测性延迟和功耗模型，联合优化预填充和解码阶段的放置策略与DVFS参数。在粗时间尺度上，DualScale计算相位感知的放置方案与基础频率，在满足SLO约束的前提下最小化能耗。在细时间尺度上，DualScale采用阶段特异性控制对每次迭代动态调整GPU频率：针对预填充阶段，采用模型预测控制（MPC）以考虑队列演化及其对未来TTFT的影响；针对解码阶段，采用轻量级松弛感知自适应方法以利用其更平滑、受内存约束的动态特性。这种分层设计能够在维持严格服务SLO的同时，实现跨时间尺度的协同控制。在面向Llama 3.3 70B模型、采用16块H100集群和生产级负载跟踪的评估中，DualScale在满足TTFT/TPOT SLO的同时，相比DistServe在预填充阶段能耗降低最高39%，解码阶段降低最高48%。