Embedding-based retrieval methods construct vector indices to search for document representations that are most similar to the query representations. They are widely used in document retrieval due to low latency and decent recall performance. Recent research indicates that deep retrieval solutions offer better model quality, but are hindered by unacceptable serving latency and the inability to support document updates. In this paper, we aim to enhance the vector index with end-to-end deep generative models, leveraging the differentiable advantages of deep retrieval models while maintaining desirable serving efficiency. We propose Model-enhanced Vector Index (MEVI), a differentiable model-enhanced index empowered by a twin-tower representation model. MEVI leverages a Residual Quantization (RQ) codebook to bridge the sequence-to-sequence deep retrieval and embedding-based models. To substantially reduce the inference time, instead of decoding the unique document ids in long sequential steps, we first generate some semantic virtual cluster ids of candidate documents in a small number of steps, and then leverage the well-adapted embedding vectors to further perform a fine-grained search for the relevant documents in the candidate virtual clusters. We empirically show that our model achieves better performance on the commonly used academic benchmarks MSMARCO Passage and Natural Questions, with comparable serving latency to dense retrieval solutions.

翻译：基于嵌入的检索方法构建向量索引，以搜索与查询表示最相似的文档表示。由于低延迟和良好的召回性能，这些方法被广泛用于文档检索。近期研究表明，深度检索解决方案能提供更好的模型质量，但受限于无法接受的推理延迟以及无法支持文档更新。本文旨在通过端到端深度生成模型增强向量索引，在保持理想服务效率的同时，利用深度检索模型的可微分优势。我们提出模型增强的向量索引（MEVI），这是一种由双塔表示模型驱动的可微分增强索引。MEVI利用残差量化码本在序列到序列深度检索与基于嵌入的模型之间建立桥梁。为大幅减少推理时间，我们并非通过长序列步骤解码唯一文档ID，而是先以少量步骤生成候选文档的语义虚拟簇ID，随后利用经过良好适应的嵌入向量在候选虚拟簇中对相关文档进行细粒度搜索。实验表明，我们的模型在常用学术基准MSMARCO Passage和Natural Questions上取得了更优性能，且服务延迟与密集检索方案相当。