The deployment of Large Language Models (LLMs) faces a critical bottleneck when handling lengthy inputs: the prohibitive memory footprint of the Key Value (KV) cache. To address this bottleneck, the token pruning paradigm leverages attention sparsity to selectively retain a small, critical subset of tokens. However, existing approaches fall short, with static methods risking irreversible information loss and dynamic strategies employing heuristics that insufficiently capture the query-dependent nature of token importance. We propose FASA, a novel framework that achieves query-aware token eviction by dynamically predicting token importance. FASA stems from a novel insight into RoPE: the discovery of functional sparsity at the frequency-chunk (FC) level. Our key finding is that a small, identifiable subset of "dominant" FCs consistently exhibits high contextual agreement with the full attention head. This provides a robust and computationally free proxy for identifying salient tokens. Building on this insight, FASA first identifies a critical set of tokens using dominant FCs, and then performs focused attention computation solely on this pruned subset. Across a spectrum of long-context tasks, from sequence modeling to complex CoT reasoning, FASA consistently outperforms all token-eviction baselines and achieves near-oracle accuracy, demonstrating remarkable robustness even under constraint budgets. Notably, on LongBench-V1, FASA reaches nearly 100\% of full-KV performance when only keeping 256 tokens, and achieves 2.56$\times$ speedup using just 18.9\% of the cache on AIME24.

翻译：大型语言模型（LLM）在处理长序列输入时面临一个关键瓶颈：键值（KV）缓存的存储开销过高。为应对这一瓶颈，令牌剪枝范式利用注意力稀疏性，选择性地保留一小部分关键令牌。然而，现有方法存在不足：静态方法可能造成不可逆的信息损失，而动态策略采用的启发式方法难以充分捕捉令牌重要性对查询的依赖性。本文提出FASA，一种通过动态预测令牌重要性实现查询感知令牌淘汰的新框架。FASA源于对RoPE的新颖洞察：在频率块（FC）层面发现了功能性稀疏性。我们的核心发现是，一小部分可识别的“主导”FC始终与完整注意力头保持高度的上下文一致性，这为识别重要令牌提供了稳健且零计算成本的代理指标。基于此发现，FASA首先利用主导FC识别关键令牌集合，随后仅在此剪枝子集上进行聚焦注意力计算。在一系列长上下文任务（从序列建模到复杂思维链推理）中，FASA始终优于所有令牌淘汰基线方法，并达到接近理论最优的准确率，即使在受限缓存预算下也展现出卓越的稳健性。值得注意的是，在LongBench-V1基准测试中，FASA仅保留256个令牌即可达到接近100%的全KV缓存性能，并在AIME24数据集上仅使用18.9%的缓存即实现2.56倍的加速。