Prefill/decode disaggregation is increasingly adopted in LLM serving to improve the latency-throughput tradeoff and meet strict TTFT and TPOT SLOs. However, LLM inference remains energy-hungry: autoscaling alone is too coarse-grained to track fast workload fluctuations, and applying fine-grained DVFS under disaggregation is complicated by phase-asymmetric dynamics and coupling between provisioning and frequency control. We present BiScale, a two-tier energy optimization framework for disaggregated LLM serving. BiScale jointly optimizes placement and DVFS across prefill and decode using predictive latency and power models. At coarse timescales, BiScale computes phase-aware placement and baseline frequencies that minimize energy while satisfying SLO constraints. At fine timescales, BiScale dynamically adapts GPU frequency per iteration using stage-specific control: model predictive control (MPC) for prefill to account for queue evolution and future TTFT impact, and lightweight slack-aware adaptation for decode to exploit its smoother, memory-bound dynamics. This hierarchical design enables coordinated control across timescales while preserving strict serving SLOs. Evaluation on a 16x H100 cluster serving Llama 3.3 70B with production-style traces shows that BiScale meets TTFT/TPOT SLOs while reducing energy by up to 39% in prefill and 48% in decode relative to DistServe.

翻译：预填充/解码解耦在大语言模型服务中日益普及，旨在改善延迟与吞吐量的权衡，并满足严格的首次令牌时间与每次输出令牌时间的服务水平目标。然而，大语言模型推理仍存在高能耗问题：仅靠自动扩缩粒度太粗，难以追踪快速的工作负载波动；而在解耦架构下应用细粒度动态电压频率调整，又因阶段不对称的动态特性以及资源供给与频率控制之间的耦合而变得复杂。本文提出BiScale，一个面向解耦式大语言模型服务的双层能量优化框架。BiScale利用预测的延迟与功耗模型，联合优化预填充和解码阶段的放置策略与动态电压频率调整。在粗粒度时间尺度上，BiScale计算阶段感知的放置方案和基准频率，在满足服务水平目标约束的同时最小化能耗。在细粒度时间尺度上，BiScale采用阶段特异性控制策略，在每次迭代中动态调整GPU频率：对预填充阶段使用模型预测控制，以考虑队列演变及对未来首次令牌时间的影响；对解码阶段则采用轻量级的松弛感知自适应策略，以利用其更平滑、内存受限的动态特性。这种分层设计实现了跨时间尺度的协调控制，同时严格保持了服务水平目标。在配备16个H100 GPU的集群上，使用生产级负载轨迹服务Llama 3.3 70B模型的评估表明，相较于DistServe，BiScale在满足首次令牌时间/每次输出令牌时间服务水平目标的同时，能将预填充阶段的能耗降低高达39%，解码阶段降低高达48%。