Serving large language models (LLMs) at scale necessitates efficient key-value (KV) cache management. KV caches can be reused across conversation turns via shared-prefix prompts that are common in iterative code editing and chat. However, stale caches consume scarce GPU memory, require offloading, or force recomputation. We present KVTC, a lightweight transform coder that compresses KV caches for compact on-GPU and off-GPU storage. Drawing on classical media compression, KVTC combines PCA-based feature decorrelation, adaptive quantization, and entropy coding. It requires only a brief initial calibration and leaves model parameters unchanged. By exploiting redundancies in KV caches, KVTC achieves up to 20$\times$ compression while maintaining reasoning and long-context accuracy, and 40$\times$ or higher for specific use cases. We test KVTC with Llama 3, Mistral NeMo, and R1-Qwen 2.5 models across benchmarks including AIME25, GSM8K, LiveCodeBench, LongBench, MATH-500, MMLU, Qasper and RULER. It consistently outperforms inference-time baselines such as token eviction, quantization, and SVD-based methods, while achieving higher compression ratios. These results support KVTC as a practical building block for memory-efficient LLM serving with reusable KV caches.

翻译：大规模服务大型语言模型（LLM）需要高效的键值（KV）缓存管理。在迭代代码编辑和对话等常见场景中，通过共享前缀提示，KV缓存可在多轮对话间重复利用。然而，陈旧的缓存会占用稀缺的GPU内存，需要卸载或强制重新计算。本文提出KVTC——一种轻量级变换编码器，通过压缩KV缓存实现GPU内外的紧凑存储。借鉴经典媒体压缩技术，KVTC结合了基于PCA的特征去相关、自适应量化和熵编码。该方法仅需短暂初始校准，且保持模型参数不变。通过挖掘KV缓存中的冗余信息，KVTC在保持推理和长上下文准确性的同时实现高达20倍的压缩比，在特定用例中可达40倍或更高。我们在Llama 3、Mistral NeMo和R1-Qwen 2.5模型上测试KVTC，涵盖AIME25、GSM8K、LiveCodeBench、LongBench、MATH-500、MMLU、Qasper和RULER等基准测试。该方法在压缩率上持续优于推理时基线方法（如令牌驱逐、量化和基于SVD的方法）。这些结果表明，KVTC可作为支持可复用KV缓存的高效内存LLM服务的实用构建模块。