While Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of complex reasoning tasks, the generated reasoning chain does not necessarily reflect how the model arrives at the answer (aka. faithfulness). We propose Faithful CoT, a faithful-by-construction framework that decomposes a reasoning task into two stages: Translation (Natural Language query $\rightarrow$ symbolic reasoning chain) and Problem Solving (reasoning chain $\rightarrow$ answer), using an LM and a deterministic solver respectively. We demonstrate the efficacy of our approach on 10 reasoning datasets from 4 diverse domains. It outperforms traditional CoT prompting on 9 out of the 10 datasets, with an average accuracy gain of 4.4 on Math Word Problems, 1.9 on Planning, 4.0 on Multi-hop Question Answering (QA), and 18.1 on Logical Inference, under greedy decoding. Together with self-consistency decoding, we achieve new state-of-the-art few-shot performance on 7 out of the 10 datasets, showing a strong synergy between faithfulness and accuracy.

翻译：尽管思维链（Chain-of-Thought, CoT）提示能提升语言模型在多种复杂推理任务上的性能，但生成的推理链并不一定反映模型得出答案的真实过程（即忠实性）。我们提出Faithful CoT，这是一种天生忠实的框架，它将推理任务分解为两个阶段：翻译（自然语言查询 → 符号推理链）和问题求解（推理链 → 答案），分别使用语言模型和确定性求解器。我们在来自4个不同领域的10个推理数据集上证明了该方法的有效性。在贪心解码下，它在10个数据集中的9个上优于传统的思维链提示，在数学应用题上平均准确率提升4.4，在规划任务上提升1.9，在多跳问答上提升4.0，在逻辑推理上提升18.1。结合自一致性解码，我们在10个数据集中的7个上实现了新的小样本最优性能，展示了忠实性与准确率之间的强协同效应。