While Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of complex reasoning tasks, the generated reasoning chain does not necessarily reflect how the model arrives at the answer (aka. faithfulness). We propose Faithful CoT, a faithful-by-construction framework that decomposes a reasoning task into two stages: Translation (Natural Language query $\rightarrow$ symbolic reasoning chain) and Problem Solving (reasoning chain $\rightarrow$ answer), using an LM and a deterministic solver respectively. We demonstrate the efficacy of our approach on 10 reasoning datasets from 4 diverse domains. It outperforms traditional CoT prompting on 9 out of the 10 datasets, with an average accuracy gain of 4.4 on Math Word Problems, 1.9 on Planning, 4.0 on Multi-hop Question Answering (QA), and 18.1 on Logical Inference, under greedy decoding. Together with self-consistency decoding, we achieve new state-of-the-art few-shot performance on 7 out of the 10 datasets, showing a strong synergy between faithfulness and accuracy.

翻译：尽管链式推理（Chain-of-Thought, CoT）提示能够提升语言模型（Language Models, LM）在一系列复杂推理任务中的表现，但生成的推理链并不一定反映模型得出答案的真实过程（即忠实性）。我们提出忠实链式推理（Faithful CoT），一种基于构造的忠实框架，将推理任务分解为两个阶段：翻译（自然语言查询 $\rightarrow$ 符号推理链）和问题求解（推理链 $\rightarrow$ 答案），分别使用语言模型和确定求解器。我们在来自4个不同领域的10个推理数据集上验证了该方法的有效性。在贪婪解码下，它在10个数据集中的9个上优于传统链式推理提示，在数学文字题上平均准确率提升4.4，在规划任务上提升1.9，在多跳问答（Multi-hop QA）上提升4.0，在逻辑推理上提升18.1。结合自一致性解码，我们在10个数据集中的7个上达到了新的少样本最优性能，展现了忠实性与准确率之间的强协同效应。