KV-cache retrieval is essential for long-context LLM inference, yet existing methods struggle with distribution drift and high latency at scale. We introduce ParisKV, a drift-robust, GPU-native KV-cache retrieval framework based on collision-based candidate selection, followed by a quantized inner-product reranking estimator. For million-token contexts, ParisKV supports CPU-offloaded KV caches via Unified Virtual Addressing (UVA), enabling on-demand top-$k$ fetching with minimal overhead. ParisKV matches or outperforms full attention quality on long-input and long-generation benchmarks. It achieves state-of-the-art long-context decoding efficiency: it matches or exceeds full attention speed even at batch size 1 for long contexts, delivers up to 2.8$\times$ higher throughput within full attention's runnable range, and scales to million-token contexts where full attention runs out of memory. At million-token scale, ParisKV reduces decode latency by 17$\times$ and 44$\times$ compared to MagicPIG and PQCache, respectively, two state-of-the-art KV-cache Top-$k$ retrieval baselines, code is available at https://github.com/amy-77/ParisKV/tree/main.

翻译：KV缓存检索对于长上下文大语言模型推理至关重要，然而现有方法在规模扩展时面临分布漂移和高延迟的挑战。我们提出ParisKV——一种基于碰撞候选选择的抗漂移、GPU原生KV缓存检索框架，并辅以量化内积重排序估计器。针对百万token级别的上下文，ParisKV通过统一虚拟地址（UVA）支持CPU卸载的KV缓存，能以最小开销实现按需top-$k$获取。在长输入和长生成基准测试中，ParisKV达到或优于全注意力机制的质量。其实现了最先进的长上下文解码效率：即便在batch size为1的长上下文场景下，解码速度亦达到或超越全注意力机制；在全注意力可运行范围内吞吐量提升高达2.8倍；并能扩展至全注意力因内存不足而无法处理的百万token上下文。在百万token规模下，与两项最先进的KV缓存Top-$k$检索基线方法MagicPIG和PQCache相比，ParisKV解码延迟分别降低17倍和44倍。代码开源于https://github.com/amy-77/ParisKV/tree/main。