Chain of Thought prompting strategy has enhanced the performance of Large Language Models (LLMs) across various NLP tasks. However, it still has shortcomings when dealing with complex reasoning tasks, following~\citet{cot_wei}, including understanding errors, calculation errors and process errors (e.g. missing-step and hallucinations). Subsequently, Our in-depth analysis of various error types has found that deeply understanding the whole problem is critical in addressing complicated reasoning tasks. In this paper, we proposed a novel prompt strategy called Deeply Understanding the Problems (DUP) prompting, inspired by how humans solve complex reasoning problems, designed to enhance the comprehensive understanding of problems by LLMs. It consists of three stages: 1) extract the core question; 2) find out problem-solving information based on the core question; 3) generate and extract answers by LLMs. We evaluate the performance of DUP prompting on ten diverse reasoning datasets. Experimental results suggest that DUP prompting significantly outperforms Zero-Shot CoT ~\cite{kojima2022large} across all datasets. Notably, DUP achieves \textbf{state-of-the-art on SVAMP (90.4\% to 94.2\%) and GSM8K (94.6\% to 97.1\%).}
翻译:思维链提示策略已提升了大语言模型在各种自然语言处理任务中的性能。然而,在处理复杂推理任务时,它仍存在不足,遵循~\citet{cot_wei}的研究,包括理解错误、计算错误和过程错误(如步骤缺失和幻觉)。随后,我们对各种错误类型的深入分析发现,深度理解整个问题对于解决复杂推理任务至关重要。本文提出了一种名为“深度理解问题”(DUP)的新颖提示策略,受人类解决复杂推理问题方式的启发,旨在增强大语言模型对问题的全面理解。该策略包含三个阶段:1)提取核心问题;2)基于核心问题找出问题解决信息;3)由大语言模型生成并提取答案。我们在十个不同的推理数据集上评估了DUP提示的性能。实验结果表明,DUP提示在所有数据集上显著优于零样本CoT~\cite{kojima2022large}。值得注意的是,DUP在SVAMP(从90.4%提升至94.2%)和GSM8K(从94.6%提升至97.1%)上实现了\textbf{最优结果。}