Long-context reasoning is a critical capability of large language models (LLMs), enabling applications such as long-document understanding, summarization, and code generation. However, efficient autoregressive inference relies on the key-value (KV) cache, whose memory footprint grows linearly with sequence length, leading to a major memory bottleneck. To mitigate this overhead, KV cache pruning methods discard cached tokens with low attention scores during inference. Most existing methods apply a uniform pruning ratio across layers, implicitly assuming that all layers contribute equally to overall model performance. We show that this assumption is suboptimal, as layers differ significantly in their sensitivity to pruning. We propose DepthKV, a layer-dependent pruning framework that allocates a fixed global KV budget across layers based on their sensitivity, rather than using a uniform allocation. Across multiple models and tasks, DepthKV consistently outperforms uniform pruning at the same global pruning ratio, demonstrating more effective utilization of the KV cache budget through layer-dependent allocation.

翻译：长上下文推理是大语言模型的一项关键能力，支撑着长文档理解、摘要生成及代码生成等应用。然而，高效的自回归推理依赖于键值缓存，其内存占用随序列长度线性增长，导致严重的内存瓶颈。为缓解这一开销，KV缓存剪枝方法在推理过程中丢弃注意力分数较低的缓存令牌。现有方法大多对各层采用统一的剪枝比例，隐含假设所有层对模型整体性能贡献相同。本文证明该假设并非最优，因为不同层对剪枝的敏感度存在显著差异。我们提出DepthKV——一种基于层级依赖的剪枝框架，根据各层敏感度在全局KV预算固定的情况下进行分配，而非采用统一分配方案。跨多项模型与任务的实验表明，在相同的全局剪枝比例下，DepthKV始终优于统一剪枝方法，通过层级依赖的分配更有效地利用了KV缓存预算。