Retrieval-augmented language models (RALMs) have recently shown great potential in mitigating the limitations of implicit knowledge in LLMs, such as untimely updating of the latest expertise and unreliable retention of long-tail knowledge. However, since the external knowledge base, as well as the retriever, can not guarantee reliability, potentially leading to the knowledge retrieved not being helpful or even misleading for LLM generation. In this paper, we introduce Supportiveness-based Knowledge Rewriting (SKR), a robust and pluggable knowledge rewriter inherently optimized for LLM generation. Specifically, we introduce the novel concept of "supportiveness"--which represents how effectively a knowledge piece facilitates downstream tasks--by considering the perplexity impact of augmented knowledge on the response text of a white-box LLM. Based on knowledge supportiveness, we first design a training data curation strategy for our rewriter model, effectively identifying and filtering out poor or irrelevant rewrites (e.g., with low supportiveness scores) to improve data efficacy. We then introduce the direct preference optimization (DPO) algorithm to align the generated rewrites to optimal supportiveness, guiding the rewriter model to summarize augmented content that better improves the final response. Comprehensive evaluations across six popular knowledge-intensive tasks and four LLMs have demonstrated the effectiveness and superiority of SKR. With only 7B parameters, SKR has shown better knowledge rewriting capability over GPT-4, the current state-of-the-art general-purpose LLM.

翻译：检索增强语言模型（RALMs）近期在缓解大语言模型（LLMs）隐含知识局限方面展现出巨大潜力，例如最新专业知识的更新滞后以及长尾知识保留的不可靠性。然而，由于外部知识库及检索器均无法保证可靠性，可能导致检索到的知识对LLM生成无益甚至产生误导。本文提出基于支持度的知识重写（SKR），这是一种为大语言模型生成任务本质优化的鲁棒可插拔知识重写器。具体而言，我们通过考量增强知识对白盒LLM响应文本的困惑度影响，提出了“支持度”这一新概念——其表征知识片段促进下游任务的有效程度。基于知识支持度，我们首先为重写器模型设计了训练数据筛选策略，通过有效识别并过滤低质量或不相关的重写结果（例如支持度评分较低者）以提升数据效能。随后引入直接偏好优化（DPO）算法，将生成的重写内容与最优支持度对齐，引导重写器模型总结出更能提升最终响应的增强内容。在六项主流知识密集型任务和四种LLM上的综合评估验证了SKR的有效性与优越性。仅凭70亿参数，SKR已展现出优于当前最先进通用大语言模型GPT-4的知识重写能力。