Retrieval-augmented generation (RAG) has emerged as a promising solution for mitigating hallucinations of large language models (LLMs) with retrieved external knowledge. Adaptive RAG enhances this approach by dynamically assessing the retrieval necessity, aiming to balance external and internal knowledge usage. However, existing adaptive RAG methods primarily realize retrieval on demand by relying on superficially verbalize-based or probability-based feedback of LLMs, or directly fine-tuning LLMs via carefully crafted datasets, resulting in unreliable retrieval necessity decisions, heavy extra costs, and sub-optimal response generation. We present the first attempts to delve into the internal states of LLMs to mitigate such issues by introducing an effective probe-guided adaptive RAG framework, termed CtrlA. Specifically, CtrlA employs an honesty probe to regulate the LLM's behavior by manipulating its representations for increased honesty, and a confidence probe to monitor the internal states of LLM and assess confidence levels, determining the retrieval necessity during generation. Experiments show that CtrlA is superior to existing adaptive RAG methods on a diverse set of tasks, the honesty control can effectively make LLMs more honest and confidence monitoring is proven to be a promising indicator of retrieval trigger. Our codes are available at https://github.com/HSLiu-Initial/CtrlA.git.

翻译：检索增强生成（RAG）已成为一种利用检索到的外部知识来缓解大语言模型（LLM）幻觉问题的有效方案。自适应RAG通过动态评估检索必要性来增强该方法，旨在平衡外部知识与内部知识的使用。然而，现有的自适应RAG方法主要依赖LLM基于表层语言描述或概率的反馈来实现按需检索，或通过精心构建的数据集直接微调LLM，这导致检索必要性决策不可靠、额外开销巨大且响应生成效果欠佳。为缓解这些问题，我们首次尝试深入探究LLM的内部状态，并提出一种有效的探针引导自适应RAG框架，命名为CtrlA。具体而言，CtrlA采用一个诚实性探针通过操控LLM的表征以增强其诚实性来调节模型行为，同时使用一个置信度探针来监测LLM的内部状态并评估置信水平，从而在生成过程中判定检索需求。实验表明，CtrlA在多种任务上均优于现有的自适应RAG方法，诚实性控制能有效提升LLM的诚实度，而置信度监测被证明是检索触发的有效指示器。我们的代码已公开于 https://github.com/HSLiu-Initial/CtrlA.git。