Recently, Chain-of-Thought (CoT) prompting has delivered success on complex reasoning tasks, which aims at designing a simple prompt like ``Let's think step by step'' or multiple in-context exemplars with well-designed rationales to elicit Large Language Models (LLMs) to generate intermediate reasoning steps. However, the generated rationales often come with mistakes, making unfactual and unfaithful reasoning chains. To mitigate this brittleness, we propose a novel Chain-of-Knowledge (CoK) prompting, where we aim at eliciting LLMs to generate explicit pieces of knowledge evidence in the form of structure triple. This is inspired by our human behaviors, i.e., we can draw a mind map or knowledge map as the reasoning evidence in the brain before answering a complex question. Benefiting from CoK, we additionally introduce a F^2-Verification method to estimate the reliability of the reasoning chains in terms of factuality and faithfulness. For the unreliable response, the wrong evidence can be indicated to prompt the LLM to rethink. Extensive experiments demonstrate that our method can further improve the performance of commonsense, factual, symbolic, and arithmetic reasoning tasks.

翻译：近年来，思维链（CoT）提示方法在复杂推理任务中取得了成功，其旨在通过设计简单的提示（如“让我们逐步思考”）或包含精心设计推理依据的上下文示例，来引导大语言模型生成中间推理步骤。然而，生成的推理依据常存在错误，导致推理链缺乏事实性与可信度。为缓解这一脆弱性，我们提出了一种新颖的知识链（CoK）提示方法，旨在引导大语言模型以结构化三元组的形式生成显式的知识证据片段。这一方法受人类行为启发——我们在回答复杂问题前，可在脑海中绘制思维导图或知识图谱作为推理证据。得益于CoK方法，我们进一步引入了F^2验证方法，从事实性与可信度两个维度评估推理链的可靠性。对于不可靠的响应，可指出错误证据以提示大语言模型重新思考。大量实验表明，我们的方法能进一步提升常识推理、事实推理、符号推理及算术推理任务的性能。