Growing context lengths in transformer-based language models have made the key-value (KV) cache a critical inference bottleneck. While many KV cache pruning methods have been proposed, they have not yet been adopted in major inference engines due to speed--accuracy trade-offs. We introduce KVzap, a fast, input-adaptive approximation of KVzip that works in both prefilling and decoding. On Qwen3-8B, Llama-3.1-8B-Instruct, and Qwen3-32B across long-context and reasoning tasks, KVzap achieves $2$--$4\times$ KV cache compression with negligible accuracy loss and achieves state-of-the-art performance on the KVpress leaderboard. Code and models are available at https://github.com/NVIDIA/kvpress.

翻译：基于Transformer的语言模型不断增长的上下文长度，使得键值（KV）缓存成为推理过程中的关键瓶颈。尽管已经提出了许多KV缓存剪枝方法，但由于速度与精度之间的权衡，它们尚未被主流推理引擎广泛采用。我们提出了KVzap，一种快速、输入自适应的KVzip近似方法，可同时应用于预填充和解码阶段。在Qwen3-8B、Llama-3.1-8B-Instruct和Qwen3-32B模型上，针对长上下文和推理任务，KVzap实现了2至4倍的KV缓存压缩，且精度损失可忽略不计，并在KVpress基准测试中取得了最先进的性能。代码与模型发布于 https://github.com/NVIDIA/kvpress。