KV-cache retrieval is essential for long-context LLM inference, yet existing methods struggle with distribution drift and high latency at scale. We introduce ParisKV, a drift-robust, GPU-native KV-cache retrieval framework based on collision-based candidate selection, followed by a quantized inner-product reranking estimator. For million-token contexts, ParisKV supports CPU-offloaded KV caches via Unified Virtual Addressing (UVA), enabling on-demand top-$k$ fetching with minimal overhead. ParisKV matches or outperforms full attention quality on long-input and long-generation benchmarks. It achieves state-of-the-art long-context decoding efficiency: it matches or exceeds full attention speed even at batch size 1 for long contexts, delivers up to 2.8$\times$ higher throughput within full attention's runnable range, and scales to million-token contexts where full attention runs out of memory. At million-token scale, ParisKV reduces decode latency by 17$\times$ and 44$\times$ compared to MagicPIG and PQCache, respectively, two state-of-the-art KV-cache Top-$k$ retrieval baselines.

翻译：键值缓存检索对于长上下文大语言模型推理至关重要，然而现有方法在大规模应用中面临分布漂移和高延迟问题。本文提出ParisKV，一种基于碰撞候选选择与量化内积重排序估计器的抗漂移、GPU原生键值缓存检索框架。针对百万令牌级别的上下文，ParisKV通过统一虚拟寻址技术支持CPU卸载的键值缓存，实现按需Top-$k$获取且开销极低。在长输入和长生成基准测试中，ParisKV达到或超越了全注意力机制的质量水平。它在长上下文解码效率方面达到最先进水平：即使在批次大小为1的长上下文场景中，其速度也达到或超过全注意力机制；在全注意力机制可运行范围内，吞吐量最高提升2.8$\times$；并能扩展至全注意力机制内存溢出的百万令牌级上下文。在百万令牌规模下，与两种最先进的键值缓存Top-$k$检索基线方法MagicPIG和PQCache相比，ParisKV分别将解码延迟降低17$\times$和44$\times$。