Multi-vector retrieval (MVR) models, exemplified by ColBERT, have established new benchmarks in retrieval accuracy by preserving fine-grained token-level interactions. However, this granularity imposes prohibitive storage and retrieval efficiency bottlenecks: to manage the immense memory footprint and computational overhead of billion-scale token vectors, state-of-the-art systems are forced to rely on aggressive dimension reduction and complex clustering (e.g., K-means). This compromise introduces two critical limitations: excessive indexing latency of clustering large-scale corpora and semantic information loss inherent to compression. In this paper, we propose Single-stage Sparse Retrieval (SSR}, a paradigm shift that replaces expensive clustering with efficient sparse coding. Instead of compressing features into low-dimensional dense vectors, we utilize Sparse Autoencoder (SAE) to project token embeddings into a high-dimensional but highly sparse representation. This transformation enables us to bypass vector clustering entirely and leverage inverted indexing for precise, high-throughput retrieval. Extensive experiments on the BEIR benchmark demonstrate that SSR achieves a "trifecta" of improvements: it reduces indexing time by 15x compared to ColBERTv2, halves retrieval latency, and simultaneously improves retrieval performance over leading baselines.
翻译:多向量检索(MVR)模型(以ColBERT为代表)通过保留细粒度的词元级交互,在检索精度上树立了新的标杆。然而,这种细粒度带来了存储和检索效率方面难以承受的瓶颈:为了管理十亿级词元向量的巨大内存占用和计算开销,现有最先进的系统被迫依赖激进的降维和复杂的聚类方法(如K-means)。这种折衷方案引入了两个关键限制:大规模语料库聚类的索引延迟过长,以及压缩过程固有的语义信息损失。本文提出单阶段稀疏检索(SSR),这是一种用高效稀疏编码取代昂贵聚类的范式转变。我们不再将特征压缩为低维稠密向量,而是利用稀疏自编码器(SAE)将词元嵌入投影到高维但高度稀疏的表示中。这种转换使我们能够完全绕过向量聚类,并利用倒排索引实现精确、高吞吐量的检索。在BEIR基准上的大量实验表明,SSR实现了三项改进的“三重奏”:与ColBERTv2相比,索引时间减少了15倍,检索延迟减半,同时检索性能相较于领先基线有所提升。