When a large language model (LLM) performs complex reasoning by chain of thought (CoT), it can be highly sensitive to individual mistakes. We have had to train verifiers to address this issue. As we all know, after human inferring a conclusion, they often check it by re-verifying it, which can avoid some mistakes. We propose a new method called self-verification that uses the conclusion of the CoT as a condition to build a new sample and asks the LLM to re-predict the original conditions which be masked. We calculate an explainable verification score based on the accuracy. This method can improve the accuracy of multiple arithmetics and logical reasoning datasets when using few-shot learning. we have demonstrated that LLMs can conduct explainable self-verification of their own conclusions and achieve competitive reasoning performance. Extensive experimentals have demonstrated that our method can help multiple large language models with self-verification can avoid interference from incorrect CoT. Code is available at \url{https://github.com/WENGSYX/Self-Verification}
翻译:当大型语言模型(LLM)通过思维链(CoT)进行复杂推理时,其性能极易受单步错误的影响。为此,我们此前需要训练验证器来应对这一问题。众所周知,人类在推演出结论后,通常会通过重新验证来检查结论,从而避免部分错误。我们提出一种名为"自我验证"的新方法,该方法以CoT的结论为条件构建新样本,要求LLM重新预测被遮蔽的原始条件,并根据准确率计算可解释的验证分数。在少样本学习场景下,该方法能显著提升多个算术与逻辑推理数据集的准确率。实验证明,LLM能够对其自身结论进行可解释的自我验证,并取得具有竞争力的推理性能。大量实验表明,我们的方法可帮助多种大型语言模型通过自我验证避免被错误CoT干扰。代码已开源至 \url{https://github.com/WENGSYX/Self-Verification}