The development of large language models (LLMs) has achieved superior performance in a range of downstream tasks, including LLM-based retrieval-augmented generation (RAG). The quality of generated content heavily relies on the usefulness of the retrieved information and the capacity of LLMs' internal information processing mechanism to incorporate it in answer generation. It is generally assumed that the retrieved information is relevant to the question. However, the retrieved information may have a variable degree of relevance and usefulness, depending on the question and the document collection. It is important to take into account the relevance of the retrieved information in answer generation. In this paper, we propose OpenDecoder, a new approach that leverages explicit evaluation of the retrieved information as quality indicator features for generation. We aim to build a RAG model that is more robust to varying levels of noisy context. Three types of explicit evaluation information are considered: relevance score, ranking score, and QPP (query performance prediction) score. The experimental results on five benchmark datasets demonstrate the effectiveness and better robustness of OpenDecoder by outperforming various baseline methods. Importantly, this paradigm is flexible to be integrated with the post-training of LLMs for any purposes and incorporated with any type of external indicators.

翻译：大型语言模型（LLM）的发展在一系列下游任务中取得了卓越性能，包括基于LLM的检索增强生成（RAG）。生成内容的质量高度依赖于检索信息的有用性以及LLM内部信息处理机制将其整合到答案生成中的能力。通常假设检索到的信息与问题相关。然而，检索到的信息可能具有不同程度的相关性和有用性，这取决于问题及文档集合。在答案生成中考虑检索信息的相关性至关重要。本文提出OpenDecoder，这是一种新方法，利用对检索信息的显式评估作为生成过程中的质量指示特征。我们的目标是构建一个对不同程度噪声上下文具有更强鲁棒性的RAG模型。我们考虑了三种显式评估信息：相关性评分、排序评分和QPP（查询性能预测）评分。在五个基准数据集上的实验结果表明，OpenDecoder通过超越多种基线方法，证明了其有效性和更优的鲁棒性。重要的是，该范式具有灵活性，可与任何目的的LLM后训练相结合，并能整合任何类型的外部指示器。