Embedding-based retrieval methods construct vector indices to search for document representations that are most similar to the query representations. They are widely used in document retrieval due to low latency and decent recall performance. Recent research indicates that deep retrieval solutions offer better model quality, but are hindered by unacceptable serving latency and the inability to support document updates. In this paper, we aim to enhance the vector index with end-to-end deep generative models, leveraging the differentiable advantages of deep retrieval models while maintaining desirable serving efficiency. We propose Model-enhanced Vector Index (MEVI), a differentiable model-enhanced index empowered by a twin-tower representation model. MEVI leverages a Residual Quantization (RQ) codebook to bridge the sequence-to-sequence deep retrieval and embedding-based models. To substantially reduce the inference time, instead of decoding the unique document ids in long sequential steps, we first generate some semantic virtual cluster ids of candidate documents in a small number of steps, and then leverage the well-adapted embedding vectors to further perform a fine-grained search for the relevant documents in the candidate virtual clusters. We empirically show that our model achieves better performance on the commonly used academic benchmarks MSMARCO Passage and Natural Questions, with comparable serving latency to dense retrieval solutions.

翻译：基于嵌入的检索方法通过构建向量索引来搜索与查询表示最相似的文档表示。由于低延迟和良好的召回性能，这些方法被广泛应用于文档检索领域。近期研究表明，深度检索方案能提供更优的模型质量，但受限于不可接受的在线服务延迟以及无法支持文档更新。本文旨在利用端到端深度生成模型增强向量索引，在保持高效在线服务能力的同时，充分发挥深度检索模型的可微分优势。我们提出模型增强型向量索引（MEVI），这是一种由双塔表示模型驱动的可微分增强索引。MEVI利用残差量化（RQ）码本桥接序列到序列深度检索与基于嵌入的模型。为显著降低推理时间，我们摒弃了以长序列步骤解码唯一文档标识符的方式，而是先通过少量步骤生成候选文档的语义虚拟聚类标识，然后利用适应性强的嵌入向量对这些候选虚拟聚类中的相关文档进行细粒度搜索。实验表明，我们的模型在常用学术基准MSMARCO Passage和Natural Questions上取得了更优性能，同时在线服务延迟与稠密检索方案相当。