In large language model (LLM) serving systems, executing each request consists of two phases: the compute-intensive prefill phase and the memory-intensive decoding phase. To prevent performance interference between the two phases, current LLM serving systems typically adopt prefill-decoding disaggregation, where the two phases are split across separate machines. However, we observe this approach leads to significant resource underutilization. Specifically, prefill instances that are compute-intensive suffer from low memory utilization, while decoding instances that are memory-intensive experience low compute utilization. To address this problem, this paper proposes Adrenaline, an attention disaggregation and offloading mechanism designed to enhance resource utilization and performance in LLM serving systems. Adrenaline's key innovation lies in disaggregating part of the attention computation in the decoding phase and offloading them to prefill instances. The memory-bound nature of decoding-phase attention computation inherently enables an effective offloading strategy, yielding two complementary advantages: 1) improved memory capacity and bandwidth utilization in prefill instances, and 2) increased decoding batch sizes that enhance compute utilization in decoding instances, collectively boosting overall system performance. Adrenaline achieves these gains through three key techniques: low-latency decoding synchronization, resource-efficient prefill colocation, and load-aware offloading scheduling. Experimental results show that Adrenaline achieves 2.28x higher memory capacity and 2.07x better memory bandwidth utilization in prefill instances, up to 1.67x improvements in compute utilization for decoding instances, and 1.68x higher overall inference throughput compared to state-of-the-art systems.

翻译：在大语言模型（LLM）服务系统中，每个请求的执行包含两个阶段：计算密集的预填充阶段和内存密集的解码阶段。为防止两阶段间的性能干扰，当前LLM服务系统通常采用预填充-解码解耦架构，将两个阶段分配到独立的机器上执行。然而，我们发现这种方法会导致严重的资源利用率不足。具体而言，计算密集的预填充实例存在内存利用率低的问题，而内存密集的解码实例则面临计算利用率不足的困境。为解决这一问题，本文提出Adrenaline——一种注意力解耦与卸载机制，旨在提升LLM服务系统的资源利用率与性能。Adrenaline的核心创新在于将解码阶段的部分注意力计算解耦并卸载至预填充实例执行。解码阶段注意力计算固有的内存受限特性，使得这种卸载策略能够有效实现两个互补优势：1) 提升预填充实例的内存容量与带宽利用率；2) 增加解码批处理规模以提升解码实例的计算利用率，从而共同提高整体系统性能。Adrenaline通过三项关键技术实现这些收益：低延迟解码同步、资源高效的预填充共置，以及负载感知的卸载调度。实验结果表明，与现有先进系统相比，Adrenaline使预填充实例的内存容量提升2.28倍、内存带宽利用率提升2.07倍，解码实例的计算利用率最高提升1.67倍，整体推理吞吐量提升1.68倍。