A core bottleneck in large language model (LLM) inference is the cost of attending over the ever-growing key-value (KV) cache. Although near-oracle top-k KV selection can preserve the quality of dense attention while sharply reducing computation and bandwidth, existing sparse methods generally rely on posterior heuristics, i.e., selectors conditioned on observed attention or proxy scores. Such conditioning introduces posterior bias: it tends to distort true token importance and miss salient tokens, thereby impairing long-range reasoning. To tackle this problem, we propose Pre-hoc Sparsity (PrHS), which selects KV entries before attention scoring and provides explicit accuracy control. Let the attention mass of discarded entries be delta (the dropped mass). Through a marginal-to-mutual-information analysis, we derive an upper bound on the mutual-information loss that depends only on the dropped mass. This relation explains failure modes of posterior heuristics and enables verifiable guarantees by controlling the dropped mass in advance. Within PrHS, we instantiate three orthogonal pre-hoc selectors along the axes of time, depth, and layer. Extensive experiments on LLaMA and Mistral families validate PrHS. Across GSM8K and CoQA, PrHS reduces retrieval overhead by over 90%, achieving 3x higher retrieval sparsity than HShare at matched or better accuracy. It incurs under 1% average degradation on LongBench, lowers attention FLOPs by about 15% versus prior sparse baselines, and yields a 9.9x speedup in attention-operator latency and 2.8x higher throughput on NVIDIA A100-80GB GPUs than the dense baseline.

翻译：大型语言模型（LLM）推理中的一个核心瓶颈在于处理不断增长的键值（KV）缓存所带来的计算成本。尽管近最优的 top-k KV 选择能够在显著减少计算和带宽的同时保持密集注意力的质量，但现有的稀疏方法通常依赖于后验启发式策略，即基于观测到的注意力分数或代理分数进行选择。这种条件化引入了后验偏差：它往往会扭曲真实的令牌重要性并遗漏显著令牌，从而损害长程推理能力。为解决此问题，我们提出了预置稀疏性（PrHS），该方法在注意力评分之前选择 KV 条目，并提供明确的精度控制。令被丢弃条目的注意力质量为 delta（丢弃质量）。通过边缘互信息分析，我们推导出互信息损失的一个上界，该上界仅依赖于丢弃质量。这一关系解释了后验启发式方法的失效模式，并通过预先控制丢弃质量实现了可验证的保证。在 PrHS 框架内，我们沿着时间、深度和层这三个正交轴实例化了三种预置选择器。在 LLaMA 和 Mistral 系列模型上的大量实验验证了 PrHS 的有效性。在 GSM8K 和 CoQA 数据集上，PrHS 将检索开销降低了超过 90%，在匹配或更优的精度下实现了比 HShare 高 3 倍的检索稀疏度。在 LongBench 上，其平均性能下降低于 1%，相较于先前的稀疏基线，注意力 FLOPs 降低了约 15%，并且在 NVIDIA A100-80GB GPU 上，注意力算子延迟实现了 9.9 倍加速，吞吐量比密集基线提高了 2.8 倍。