Recent research has shown that transformer networks can be used as differentiable search indexes by representing each document as a sequences of document ID tokens. These generative retrieval models cast the retrieval problem to a document ID generation problem for each given query. Despite their elegant design, existing generative retrieval models only perform well on artificially-constructed and small-scale collections. This has led to serious skepticism in the research community on their real-world impact. This paper represents an important milestone in generative retrieval research by showing, for the first time, that generative retrieval models can be trained to perform effectively on large-scale standard retrieval benchmarks. For doing so, we propose RIPOR- an optimization framework for generative retrieval that can be adopted by any encoder-decoder architecture. RIPOR is designed based on two often-overlooked fundamental design considerations in generative retrieval. First, given the sequential decoding nature of document ID generation, assigning accurate relevance scores to documents based on the whole document ID sequence is not sufficient. To address this issue, RIPOR introduces a novel prefix-oriented ranking optimization algorithm. Second, initial document IDs should be constructed based on relevance associations between queries and documents, instead of the syntactic and semantic information in the documents. RIPOR addresses this issue using a relevance-based document ID construction approach that quantizes relevance-based representations learned for documents. Evaluation on MSMARCO and TREC Deep Learning Track reveals that RIPOR surpasses state-of-the-art generative retrieval models by a large margin (e.g., 30.5% MRR improvements on MS MARCO Dev Set), and perform better on par with popular dense retrieval models.

翻译：近期研究表明，Transformer网络可通过将每篇文档表示为文档ID令牌序列，作为可微搜索索引使用。这类生成式检索模型将检索问题转化为针对给定查询的文档ID生成问题。尽管设计精妙，现有生成式检索模型仅在人工构建的小规模数据集上表现良好，这引发了研究界对其实际应用价值的严重质疑。本文通过首次证明生成式检索模型能够在标准大规模检索基准上实现有效训练，标志着该领域研究的重要里程碑。为此，我们提出RIPOR——一种可适配任意编码器-解码器架构的生成式检索优化框架。RIPOR的设计基于生成式检索中两个常被忽视的基础性考量：其一，鉴于文档ID生成的序贯解码特性，仅基于完整文档ID序列分配准确相关性得分并不充分。为此，RIPOR引入新型前缀导向排序优化算法；其二，初始文档ID应依据查询与文档间的关联性构建，而非文档的句法语义信息。RIPOR通过基于相关性的文档ID构建方法解决该问题，该方法对学习到的相关性表征进行量化。在MS MARCO和TREC Deep Learning Track上的评估表明，RIPOR以显著优势超越现有最优生成式检索模型（如在MS MARCO开发集上MRR提升30.5%），并与主流密集检索模型性能相当。