Large Language Models (LLMs) have exhibited exceptional performance across a broad range of tasks and domains. However, they still encounter difficulties in solving mathematical problems due to the rigorous and logical nature of mathematics. Previous studies have employed techniques such as supervised fine-tuning (SFT), prompt engineering, and search-based methods to improve the mathematical problem-solving abilities of LLMs. Despite these efforts, their performance remains suboptimal and demands substantial computational resources. To address this issue, we propose a novel approach, BEATS, to enhance mathematical problem-solving abilities. Our method leverages newly designed prompts that guide the model to iteratively rewrite, advance by one step, and generate answers based on previous steps. Additionally, we introduce a new back-verification technique that uses LLMs to validate the correctness of the generated answers. Furthermore, we employ a pruning tree search to optimize search time while achieving strong performance. Notably, our method improves Qwen2-7b-Instruct's score from 36.94 to 61.52, outperforming GPT4's 42.5 on the MATH benchmark.

翻译：大语言模型（LLMs）已在广泛的任务和领域中展现出卓越的性能。然而，由于其严格的逻辑性，数学问题求解仍是LLMs面临的挑战。先前研究采用了监督微调（SFT）、提示工程和基于搜索的方法来提升LLMs的数学解题能力。尽管付出了这些努力，其性能仍不理想且需要大量计算资源。为解决这一问题，我们提出了一种新颖方法BEATS以增强数学问题求解能力。我们的方法利用新设计的提示，引导模型基于前序步骤迭代重写、单步推进并生成答案。此外，我们引入了一种新的反向验证技术，使用LLMs来验证生成答案的正确性。进一步地，我们采用剪枝树搜索以优化搜索时间，同时实现强劲性能。值得注意的是，我们的方法将Qwen2-7b-Instruct在MATH基准上的得分从36.94提升至61.52，超越了GPT4的42.5分。