Generative retrieval models encode pointers to information in a corpus as an index within the model's parameters. These models serve as part of a larger pipeline, where retrieved information conditions generation for knowledge-intensive NLP tasks. However, we identify two limitations: the generative retrieval does not account for contextual information. Secondly, the retrieval can't be tuned for the downstream readers as decoding the page title is a non-differentiable operation. This paper introduces Re3val, trained with generative reranking and reinforcement learning using limited data. Re3val leverages context acquired via Dense Passage Retrieval to rerank the retrieved page titles and utilizes REINFORCE to maximize rewards generated by constrained decoding. Additionally, we generate questions from our pre-training dataset to mitigate epistemic uncertainty and bridge the domain gap between the pre-training and fine-tuning datasets. Subsequently, we extract and rerank contexts from the KILT database using the rerank page titles. Upon grounding the top five reranked contexts, Re3val demonstrates the Top 1 KILT scores compared to all other generative retrieval models across five KILT datasets.

翻译：生成式检索模型将指向语料库中信息的指针编码为模型参数中的索引。这些模型作为更大流程的一部分，其中检索到的信息为知识密集型自然语言处理任务的生成提供支撑。然而，我们发现了两个局限性：首先，生成式检索未考虑上下文信息；其次，由于解码页面标题是非可微操作，检索无法针对下游阅读器进行调优。本文提出了Re3val，它通过有限数据训练，结合了生成式重排序与强化学习。Re3val利用稠密段落检索获取的上下文对检索到的页面标题进行重排序，并采用REINFORCE算法最大化由约束解码产生的奖励。此外，我们从预训练数据集中生成问题以缓解认知不确定性，并弥合预训练与微调数据集之间的领域差距。随后，我们使用重排序后的页面标题从KILT数据库中提取并重排序上下文。在基于前五个重排序上下文的基础上，Re3val在五个KILT数据集上的KILT所有生成式检索模型中取得了最高分。