Large Language Models (LLMs) have driven substantial progress in artificial intelligence in recent years, exhibiting impressive capabilities across a wide range of tasks, including mathematical problem-solving. Inspired by the success of subgoal-based methods, we propose a novel framework called \textbf{SE}quential sub\textbf{G}oal \textbf{O}ptimization (SEGO) to enhance LLMs' ability to solve mathematical problems. By establishing a connection between the subgoal breakdown process and the probability of solving problems, SEGO aims to identify better subgoals with theoretical guarantees. Addressing the challenge of identifying suitable subgoals in a large solution space, our framework generates problem-specific subgoals and adjusts them according to carefully designed criteria. Incorporating these optimized subgoals into the policy model training leads to significant improvements in problem-solving performance. We validate SEGO's efficacy through experiments on two benchmarks, GSM8K and MATH, where our approach outperforms existing methods, highlighting the potential of SEGO in AI-driven mathematical problem-solving. Data and code associated with this paper will be available at https://github.com/zhaoxlpku/SEGO

翻译：近年来，大型语言模型（LLMs）推动了人工智能领域的重大进展，在包括数学问题求解在内的广泛任务中展现出卓越能力。受基于子目标方法的成功启发，我们提出了一种名为**SE**quential sub**G**oal **O**ptimization（SEGO）的新型框架，旨在提升LLMs求解数学问题的能力。通过建立子目标分解过程与问题求解概率之间的关联，SEGO旨在理论保证下识别更优的子目标。针对在庞大解空间中定位合适子目标的挑战，该框架生成问题专属子目标，并依据精心设计的标准对其进行调整。将这些优化后的子目标纳入策略模型训练，能够显著提升问题求解性能。我们在GSM8K和MATH两个基准测试上通过实验验证了SEGO的有效性，结果表明该方法优于现有技术，突显了SEGO在人工智能驱动的数学问题求解中的潜力。本文相关数据和代码将在https://github.com/zhaoxlpku/SEGO 公开。