Large Language Models (LLMs) are increasingly deployed in complex multi-agent applications that rely on external function calls. This workload creates severe performance challenges for the KV Cache: spatial contention leads to the eviction of critical agents' caches and temporal underutilization leaves the cache of agents stalled on long-running function calls idling in GPU memory. We present TokenCake, a KV-Cache-centric serving framework that bridges this gap by co-optimizing scheduling and memory management through an agent-aware design. TokenCake's Temporal Scheduler employs an event-driven, opportunistic policy to proactively offload idle KV Caches during function calls and uses predictive uploading to hide data transfer latency. TokenCake's Spatial Scheduler uses dynamic memory partitioning, guided by a hybrid priority metric combining graph structure and runtime state, to reserve GPU memory for critical-path agents. Our evaluation on representative multi-agent benchmarks shows that TokenCake reduces end-to-end latency by over 47.06% and improves effective GPU memory utilization by up to 16.9% compared to vLLM.

翻译：大型语言模型（LLM）正被日益部署于依赖外部函数调用的复杂多智能体应用中。此类工作负载对KV缓存造成了严峻的性能挑战：空间竞争导致关键智能体的缓存被逐出，而时间利用不足又使因长函数调用而阻塞的智能体缓存闲置在GPU显存中。我们提出TokenCake——一种以KV缓存为中心的服务框架，通过智能体感知的设计协同优化调度与内存管理，弥合这一差距。TokenCake的时间调度器采用事件驱动的机会主义策略，在函数调用期间主动卸载闲置KV缓存，并利用预测加载技术隐藏数据传输延迟。TokenCake的空间调度器采用动态内存分区方法，基于融合图结构与运行时状态的混合优先级指标进行引导，为关键路径上的智能体预留GPU显存。在代表性多智能体基准测试上的评估表明，与vLLM相比，TokenCake将端到端延迟降低了超过47.06%，并将有效GPU显存利用率提升高达16.9%。