Speculative decoding and dynamic sparse attention are two complementary approaches for accelerating long-context LLM inference: the former amortizes target-model execution across multiple verifier queries, while the latter reduces each query's KV-cache working set. Directly combining them, however, exposes a structural mismatch: speculative verification relies on cross-query commonality, whereas dynamic sparse attention assigns query-specific sparse layouts. This mismatch limits KV-block reuse, amplifies NSA's branch-wise overheads, and makes verification strategy selection input- and regime-dependent. We present SSV, a sparse speculative-verification framework that turns dynamic sparse attention into a verification-oriented workload. SSV combines overlap-aware grouped-query execution, refresh/reuse-based NSA kernel fusion, and profile-guided prompt-adaptive orchestration to improve cross-query reuse, reduce selected-index and branch-fusion overheads, and select effective draft-verification strategies under user-specified precision classes. Experiments on NVIDIA H100 GPUs show that SSV achieves up to 3.49x end-to-end throughput over autoregressive NSA decoding and up to 6.86x kernel speedups for sparse speculative verification.

翻译：推测解码与动态稀疏注意力是加速长上下文LLM推理的两种互补方法：前者通过多个验证器查询分摊目标模型执行开销，后者则缩减每次查询的KV缓存工作集。然而，直接结合两者会暴露出结构性错配：推测验证依赖跨查询共性，而动态稀疏注意力分配的是查询特定的稀疏布局。这种错配限制了KV块复用，放大了NSA的分支开销，并使验证策略的选择依赖于输入和运行模式。我们提出SSV，一个将动态稀疏注意力转化为面向验证工作负载的稀疏推测验证框架。SSV融合了重叠感知的成组查询执行、基于刷新/重用的NSA内核融合以及基于性能剖析的提示自适应编排，以提升跨查询复用、减少选定索引与分支融合开销，并在用户指定的精度类别下选择有效的草稿-验证策略。在NVIDIA H100 GPU上的实验表明，相较于自回归的NSA解码，SSV可实现高达3.49倍的端到端吞吐量，并为稀疏推测验证带来高达6.86倍的内核加速。