Recently, we have witnessed generative retrieval increasingly gaining attention in the information retrieval (IR) field, which retrieves documents by directly generating their identifiers. So far, much effort has been devoted to developing effective generative retrieval models. There has been less attention paid to the robustness perspective. When a new retrieval paradigm enters into the real-world application, it is also critical to measure the out-of-distribution (OOD) generalization, i.e., how would generative retrieval models generalize to new distributions. To answer this question, firstly, we define OOD robustness from three perspectives in retrieval problems: 1) The query variations; 2) The unforeseen query types; and 3) The unforeseen tasks. Based on this taxonomy, we conduct empirical studies to analyze the OOD robustness of several representative generative retrieval models against dense retrieval models. The empirical results indicate that the OOD robustness of generative retrieval models requires enhancement. We hope studying the OOD robustness of generative retrieval models would be advantageous to the IR community.

翻译：最近，我们见证了生成式检索在信息检索领域日益受到关注，它通过直接生成文档标识符来检索文档。迄今为止，大量研究致力于开发高效的生成式检索模型，但对其鲁棒性的关注相对较少。当一种新的检索范式进入实际应用时，衡量其分布外泛化能力——即生成式检索模型如何泛化到新分布——同样至关重要。为解答此问题，我们首先从检索问题的三个角度定义分布外鲁棒性：1）查询变化；2）未预见的查询类型；3）未预见的任务。基于这一分类，我们通过实证研究分析了多种代表性生成式检索模型相较于稠密检索模型的分布外鲁棒性。实验结果表明，生成式检索模型的分布外鲁棒性亟待提升。我们期望对生成式检索模型分布外鲁棒性的研究能对信息检索领域有所裨益。