Large language model (LLM) inference has become a dominant workload in modern data centers, driving significant GPU utilization and energy consumption. While prior systems optimize throughput and latency by batching, scheduling, and parallelism, they largely treat GPU power as a static constraint rather than a controllable resource. In this paper, we present a power-aware runtime for LLM serving, PALS, that treats GPU power caps as a first-class control knob and jointly optimizes them with software parameters such as batch size. The system combines lightweight offline power-performance models with a feedback-driven controller to select configurations that satisfy throughput targets while maximizing energy efficiency. We implement PALS within an existing LLM serving framework, vLLM, demonstrating that it requires no model retraining or API changes. Across multi-GPU systems and both dense and mixture-of-experts (MoE) models, PALS improves energy efficiency by up to 26.3%, reduces QoS violations by 4x to 7x under power constraints, and tracks dynamic power budgets. These results highlight the potential of integrating power control directly into LLM inference runtimes, enabling energy-proportional and grid-interactive AI systems.

翻译：大语言模型推理已成为现代数据中心的主导工作负载，驱动着显著的GPU利用率和能耗。现有系统通过批处理、调度和并行化来优化吞吐量和延迟，但通常将GPU功耗视为静态约束而非可调控资源。本文提出一种面向大语言模型服务的功耗感知运行时系统PALS，它将GPU功耗上限作为一等控制自由度，并与批次大小等软件参数进行联合优化。该系统将轻量级离线功耗性能模型与反馈驱动控制器相结合，选择满足吞吐量目标并最大化能效的配置。我们在现有大语言模型服务框架vLLM中实现了PALS，证明其无需模型重训练或API改动。在多GPU系统以及密集模型和混合专家模型上，PALS将能效提升高达26.3%，在功耗约束下将服务质量违规率降低4至7倍，并能追踪动态功耗预算。这些结果凸显了将功耗控制直接集成到LLM推理运行时中的潜力，从而实现了能源比例型和电网交互式AI系统。