Recently, Chain-of-Thought (CoT) prompting has delivered success on complex reasoning tasks, which aims at designing a simple prompt like ``Let's think step by step'' or multiple in-context exemplars with well-designed rationales to elicit Large Language Models (LLMs) to generate intermediate reasoning steps. However, the generated rationales often come with mistakes, making unfactual and unfaithful reasoning chains. To mitigate this brittleness, we propose a novel Chain-of-Knowledge (CoK) prompting, where we aim at eliciting LLMs to generate explicit pieces of knowledge evidence in the form of structure triple. This is inspired by our human behaviors, i.e., we can draw a mind map or knowledge map as the reasoning evidence in the brain before answering a complex question. Benefiting from CoK, we additionally introduce a F^2-Verification method to estimate the reliability of the reasoning chains in terms of factuality and faithfulness. For the unreliable response, the wrong evidence can be indicated to prompt the LLM to rethink. Extensive experiments demonstrate that our method can further improve the performance of commonsense, factual, symbolic, and arithmetic reasoning tasks.

翻译：近期，链式思维（CoT）提示在复杂推理任务上取得了成功，其旨在通过设计如"让我们逐步思考"等简单提示，或包含精心设计推理过程的多上下文示例，引导大型语言模型（LLM）生成中间推理步骤。然而，生成的推理过程常伴随错误，导致非事实性和不忠实的推理链。为缓解这一脆弱性，我们提出了一种新颖的基于知识链（CoK）提示方法，旨在引导大型语言模型以结构化三元组形式生成显式的知识证据。这一方法受人类行为启发，即我们在回答复杂问题前，可在脑海中绘制思维导图或知识图谱作为推理证据。借助CoK，我们进一步引入F²验证方法，从事实性和忠实性角度评估推理链的可靠性。对于不可靠的响应，可指出错误证据以提示大型语言模型重新思考。大量实验表明，我们的方法能够进一步提升常识推理、事实推理、符号推理和算术推理任务的性能。