Benchmarking the performance of information retrieval (IR) is mostly conducted with a fixed set of documents (static corpora). However, in realistic scenarios, this is rarely the case and the documents to be retrieved are constantly updated and added. In this paper, we focus on Generative Retrievals (GR), which apply autoregressive language models to IR problems, and explore their adaptability and robustness in dynamic scenarios. We also conduct an extensive evaluation of computational and memory efficiency, crucial factors for real-world deployment of IR systems handling vast and ever-changing document collections. Our results on the StreamingQA benchmark demonstrate that GR is more adaptable to evolving knowledge (4 -- 11%), robust in learning knowledge with temporal information, and efficient in terms of inference FLOPs (x 2), indexing time (x 6), and storage footprint (x 4) compared to Dual Encoders (DE), which are commonly used in retrieval systems. Our paper highlights the potential of GR for future use in practical IR systems within dynamic environments.

翻译：信息检索（IR）的性能评估大多基于固定文档集（静态语料库）进行。然而，在实际场景中，这种情况很少见，待检索的文档会不断更新和添加。本文聚焦于生成式检索（GR）——该方法将自回归语言模型应用于IR问题，并探讨其在动态场景中的适应性和鲁棒性。我们还对计算和内存效率进行了广泛评估，这对于处理海量且不断变化的文档集合的实际IR系统部署至关重要。我们在StreamingQA基准测试上的结果表明，与检索系统中常用的双编码器（DE）相比，GR对演进知识的适应性更强（4-11%），在学习具有时间信息的知识时更鲁棒，并且在推理FLOPs（×2）、索引时间（×6）和存储占用（×4）方面更高效。本文强调了GR在未来动态环境下的实际IR系统中应用的潜力。