Knowledge graph-based retrieval-augmented generation seeks to mitigate hallucinations in Large Language Models (LLMs) caused by insufficient or outdated knowledge. However, existing methods often fail to fully exploit the prior knowledge embedded in knowledge graphs (KGs), particularly their structural information and explicit or implicit constraints. The former can enhance the faithfulness of LLMs' reasoning, while the latter can improve the reliability of response generation. Motivated by these, we propose a trustworthy reasoning framework, termed Deliberation over Priors (DP), which sufficiently utilizes the priors contained in KGs. Specifically, DP adopts a progressive knowledge distillation strategy that integrates structural priors into LLMs through a combination of supervised fine-tuning and Kahneman-Tversky optimization, thereby improving the faithfulness of relation path generation. Furthermore, our framework employs a reasoning-introspection strategy, which guides LLMs to perform refined reasoning verification based on extracted constraint priors, ensuring the reliability of response generation. Extensive experiments on three benchmark datasets demonstrate that DP achieves new state-of-the-art performance, especially a Hit@1 improvement of 13% on the ComplexWebQuestions dataset, and generates highly trustworthy responses. We also conduct various analyses to verify its flexibility and practicality. The code is available at https://github.com/reml-group/Deliberation-on-Priors.

翻译：基于知识图谱的检索增强生成旨在缓解大语言模型因知识不足或过时而产生的幻觉问题。然而，现有方法往往未能充分利用知识图谱中蕴含的先验知识，特别是其结构信息与显式或隐式约束。前者可增强大语言模型推理的忠实性，而后者能提升响应生成的可靠性。基于此，我们提出一个名为先验审思的可信推理框架，该框架充分挖掘知识图谱中的先验信息。具体而言，DP采用渐进式知识蒸馏策略，通过监督微调与Kahneman-Tversky优化的结合，将结构先验融入大语言模型，从而提升关系路径生成的忠实度。此外，本框架采用推理-内省策略，引导大语言模型基于提取的约束先验进行精细化推理验证，确保响应生成的可靠性。在三个基准数据集上的大量实验表明，DP取得了最新的最优性能，尤其在ComplexWebQuestions数据集上实现了13%的Hit@1提升，并能生成高可信度的响应。我们还通过多维度分析验证了其灵活性与实用性。代码已开源：https://github.com/reml-group/Deliberation-on-Priors。