Retrieval-enhanced text generation, which aims to leverage passages retrieved from a large passage corpus for delivering a proper answer given the input query, has shown remarkable progress on knowledge-intensive language tasks such as open-domain question answering and knowledge-enhanced dialogue generation. However, the retrieved passages are not ideal for guiding answer generation because of the discrepancy between retrieval and generation, i.e., the candidate passages are all treated equally during the retrieval procedure without considering their potential to generate the proper answers. This discrepancy makes a passage retriever deliver a sub-optimal collection of candidate passages to generate answers. In this paper, we propose the GeneRative Knowledge Improved Passage Ranking (GripRank) approach, addressing the above challenge by distilling knowledge from a generative passage estimator (GPE) to a passage ranker, where the GPE is a generative language model used to measure how likely the candidate passages can generate the proper answer. We realize the distillation procedure by teaching the passage ranker learning to rank the passages ordered by the GPE. Furthermore, we improve the distillation quality by devising a curriculum knowledge distillation mechanism, which allows the knowledge provided by the GPE can be progressively distilled to the ranker through an easy-to-hard curriculum, enabling the passage ranker to correctly recognize the provenance of the answer from many plausible candidates. We conduct extensive experiments on four datasets across three knowledge-intensive language tasks. Experimental results show advantages over the state-of-the-art methods for both passage ranking and answer generation on the KILT benchmark.

翻译：检索增强文本生成旨在利用从大型语料库中检索到的段落为输入查询提供恰当答案，已在开放域问答、知识增强对话生成等知识密集型语言任务中取得显著进展。然而，由于检索与生成之间存在差异（即检索过程中所有候选段落被平等对待，未考虑其生成正确答案的潜力），检索到的段落难以有效指导答案生成。这种差异导致段落检索器向答案生成阶段传递次优的候选段落集合。本文提出生成式知识改进段落排序方法，通过将生成式段落评估器的知识蒸馏至段落排序器来解决上述挑战。该评估器采用生成式语言模型衡量候选段落生成正确答案的可行性。我们通过教导段落排序器学习按生成式段落评估器排序的段落顺序来实现蒸馏过程。进一步地，我们设计课程知识蒸馏机制提升蒸馏质量，该机制使生成式段落评估器提供的知识能通过由易到难的课程逐步蒸馏至排序器，从而帮助段落排序器从大量候选段落中准确识别答案来源。我们在涵盖三个知识密集型语言任务的四个数据集上开展广泛实验，结果表明本方法在KILT基准测试的段落排序和答案生成任务上均优于现有最优方法。