The integration of large language models (LLMs) and search engines represents a significant evolution in knowledge acquisition methodologies. However, determining the knowledge that an LLM already possesses and the knowledge that requires the help of a search engine remains an unresolved issue. Most existing methods solve this problem through the results of preliminary answers or reasoning done by the LLM itself, but this incurs excessively high computational costs. This paper introduces a novel collaborative approach, namely SlimPLM, that detects missing knowledge in LLMs with a slim proxy model, to enhance the LLM's knowledge acquisition process. We employ a proxy model which has far fewer parameters, and take its answers as heuristic answers. Heuristic answers are then utilized to predict the knowledge required to answer the user question, as well as the known and unknown knowledge within the LLM. We only conduct retrieval for the missing knowledge in questions that the LLM does not know. Extensive experimental results on five datasets with two LLMs demonstrate a notable improvement in the end-to-end performance of LLMs in question-answering tasks, achieving or surpassing current state-of-the-art models with lower LLM inference costs.

翻译：大型语言模型（LLM）与搜索引擎的整合代表了知识获取方法的重大演进。然而，如何判定LLM已掌握的知识与仍需借助搜索引擎获取的知识，仍是一个悬而未决的问题。现有方法大多通过LLM自身生成的初步答案或推理结果来解决此问题，但这会带来极高的计算成本。本文提出一种新颖的协同方法——SlimPLM，即通过一个轻量代理模型来检测LLM中缺失的知识，以优化其知识获取过程。我们采用一个参数规模远小于LLM的代理模型，并将其生成的答案作为启发式答案。这些启发式答案随后被用于预测回答用户问题所需的知识，以及LLM内部已知与未知的知识领域。我们仅针对LLM未知问题中的缺失知识进行检索。在五个数据集及两种LLM上的大量实验结果表明，该方法显著提升了LLM在问答任务中的端到端性能，以更低的LLM推理成本达到或超越了当前最先进模型的水平。