While dense retrieval models, which embed queries and documents into a shared low-dimensional space, have gained widespread popu- larity, they were shown to exhibit important theoretical limitations and considerably lag behind traditional sparse retrieval models in certain settings. Generative retrieval has emerged as an alternative approach to dense retrieval by using a language model to predict query-document relevance directly. In this paper, we demonstrate strengths and weaknesses of generative retrieval approaches us- ing a simple synthetic dataset, called LIMIT, that was previously introduced to empirically demonstrate the theoretical limitations of embedding-based retrieval but was not used to evaluate genera- tive retrieval. We close this research gap and show that generative retrieval achieves the best performance on this dataset without any additional training required (0.92 and 0.99 R@2 for SEAL and MINDER, respectively), compared to dense approaches (< 0.03 Re- call@2) and BM25 (0.86 R@2). However, we then proceed to extend the original LIMIT dataset by adding simple hard negative samples and observe the performance degrading for all the models including the generative retrieval models (0.51 R@2) as well as BM25 (0.21 R@2). Error analysis identifies a failure in the decoding mechanism, caused by the inability to produce identifiers that are unique to relevant documents. Future generative retrieval must address these issues, either by designing identifiers that are more suitable to the decoding process or by adapting decoding and scoring algorithms to preserve relevance signals.

翻译：虽然将查询和文档嵌入共享低维空间的稠密检索模型已广泛普及，但研究表明其存在重要的理论局限性，并在某些场景下明显落后于传统稀疏检索模型。生成式检索通过利用语言模型直接预测查询与文档的相关性，成为稠密检索的替代方案。本文利用名为LIMIT的简单合成数据集，展示了生成式检索方法的优势与不足。该数据集此前被用于实证展示基于嵌入的检索的理论局限性，但尚未用于评估生成式检索。我们填补了这一研究空白，并证明：在该数据集上，生成式检索无需额外训练即可达到最优性能（SEAL和MINDER的R@2分别为0.92和0.99），而稠密方法（Recall@2<0.03）和BM25（R@2=0.86）均不及。然而，我们进一步扩展原始LIMIT数据集，加入简单难负样本后，观察到所有模型性能均出现下降，包括生成式检索模型（R@2=0.51）和BM25（R@2=0.21）。错误分析揭示了解码机制的失效根源——模型无法生成与相关文档唯一对应的标识符。未来生成式检索必须解决这些问题，要么设计更适配解码过程的标识符，要么调整解码与评分算法以保留相关性信号。