This study presents a novel learning approach designed to enhance both mathematical reasoning and problem-solving abilities of Large Language Models (LLMs). We focus on integrating the Chain-of-Thought (CoT) and the Program-of-Thought (PoT) learning, hypothesizing that prioritizing the learning of mathematical reasoning ability is helpful for the amplification of problem-solving ability. Thus, the initial learning with CoT is essential for solving challenging mathematical problems. To this end, we propose a sequential learning approach, named SAAS (Solving Ability Amplification Strategy), which strategically transitions from CoT learning to PoT learning. Our empirical study, involving an extensive performance comparison using several benchmarks, demonstrates that our SAAS achieves state-of-the-art (SOTA) performance. The results underscore the effectiveness of our sequential learning approach, marking a significant advancement in the field of mathematical reasoning in LLMs.

翻译：本研究提出一种新颖的学习方法，旨在同时提升大语言模型的数学推理与问题解决能力。我们聚焦于整合思维链学习与程序思维学习，并提出以下假设：优先学习数学推理能力有助于放大问题解决能力。因此，初始的思维链学习对于解决复杂数学问题至关重要。为此，我们提出一种名为SAAS的顺序学习方法，该方法策略性地从思维链学习过渡至程序思维学习。我们通过使用多个基准测试进行广泛性能比较的实证研究表明，所提出的SAAS方法取得了最先进的性能。该结果凸显了我们顺序学习方法的有效性，标志着大语言模型数学推理领域的重要进展。