Large Language Models (LLMs) rely on optimizations like Automatic Prefix Caching (APC) to accelerate inference. APC works by reusing previously computed states for the beginning part of a request (prefix), when another request starts with the same text. While APC improves throughput, it introduces timing side channels: cache hits are faster than misses, creating observable latency differences. In multi-tenant systems, attackers can exploit these differences to infer sensitive information, e.g., by incrementally reconstructing another user's request by observing hit/miss patterns. Current defenses take a sledgehammer approach: they disable APC and cache sharing, isolating users, and sacrificing efficiency for regular users. This paper presents CacheSolidarity, a system that secures multi-tenant LLM serving systems against APC side channels without sacrificing performance and efficiency. CacheSolidarity monitors cache reuse across users, flags suspicious sharing, and selectively isolates prefixes, restricting their reuse only when necessary. Evaluation shows that CacheSolidarity enables up to 70% higher cache reuse and 30% lower inference latency compared to existing defenses that isolate users. CacheSolidarity's lightweight design demonstrates how security in LLM serving does not have to come at the cost of unnecessarily reduced performance or unbearable overheads.

翻译：大型语言模型（LLM）依赖自动前缀缓存（APC）等优化技术来加速推理。APC的工作原理是：当另一个请求以相同文本开头时，复用先前为请求起始部分（前缀）计算的状态。虽然APC提升了吞吐量，但也引入了时序侧信道：缓存命中的速度远快于未命中，从而产生可观测的延迟差异。在多租户系统中，攻击者可利用这些差异推断敏感信息，例如通过观察命中/未命中模式逐步重构其他用户的请求。现有防御方案采取“一刀切”策略：禁用APC与缓存共享，隔离用户，并以牺牲普通用户效率为代价。本文提出CacheSolidarity系统，可在不牺牲性能与效率的前提下，保护多租户LLM服务系统免受APC侧信道攻击。CacheSolidarity监控跨用户的缓存复用行为，标记可疑共享，并选择性隔离前缀——仅在必要时限制其复用。评估表明，与现有隔离用户的防御方案相比，CacheSolidarity可实现高达70%的缓存复用率提升和30%的推理延迟降低。其轻量化设计证明，LLM服务系统的安全性无需以不必要的性能下降或难以承受的开销为代价。