This study presents a novel learning approach designed to enhance both mathematical reasoning and problem-solving abilities of Large Language Models (LLMs). We focus on integrating the Chain-of-Thought (CoT) and the Program-of-Thought (PoT) learning, hypothesizing that prioritizing the learning of mathematical reasoning ability is helpful for the amplification of problem-solving ability. Thus, the initial learning with CoT is essential for solving challenging mathematical problems. To this end, we propose a sequential learning approach, named SAAS (Solving Ability Amplification Strategy), which strategically transitions from CoT learning to PoT learning. Our empirical study, involving an extensive performance comparison using several benchmarks, demonstrates that our SAAS achieves state-of-the-art (SOTA) performance. The results underscore the effectiveness of our sequential learning approach, marking a significant advancement in the field of mathematical reasoning in LLMs.

翻译：本研究提出了一种旨在增强大语言模型（LLMs）数学推理与问题求解能力的新型学习方法。我们专注于整合思维链（CoT）与程序链（PoT）学习，假设优先学习数学推理能力有助于问题求解能力的放大。因此，初始阶段的CoT学习对于解决具有挑战性的数学问题至关重要。为此，我们提出一种名为SAAS（求解能力放大策略）的序列学习方法，该方法战略性地实现从CoT学习到PoT学习的过渡。我们的实证研究在多个基准测试上进行了广泛的性能比较，结果表明SAAS达到了最先进的（SOTA）性能。这些结果凸显了序列学习方法的有效性，标志着LLMs数学推理领域的重要进展。