Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical problems by seamlessly integrating natural language reasoning with the utilization of external tools (e.g., computation libraries and symbolic solvers), thereby amalgamating the analytical prowess of language and the computational efficiency of tools. To train ToRA, we curate interactive tool-use trajectories on mathematical datasets, apply imitation learning on the annotations, and propose output space shaping to further refine models' reasoning behavior. As a result, ToRA models significantly outperform open-source models on 10 mathematical reasoning datasets across all scales with 13%-19% absolute improvements on average. Notably, ToRA-7B reaches 44.6% on the competition-level dataset MATH, surpassing the best open-source model WizardMath-70B by 22% absolute. ToRA-Code-34B is also the first open-source model that achieves an accuracy exceeding 50% on MATH, which significantly outperforms GPT-4's CoT result, and is competitive with GPT-4 solving problems with programs. Additionally, we conduct a comprehensive analysis of the benefits and remaining challenges of tool interaction for mathematical reasoning, providing valuable insights for future research.

翻译：大语言模型在各种语言任务中取得了显著进展，但在复杂数学问题上仍存在不足。本文提出ToRA系列工具集成推理智能体，通过将自然语言推理与外部工具（如计算库和符号求解器）无缝集成，融合语言的分析能力与工具的计算效率，以求解具有挑战性的数学问题。为训练ToRA，我们在数学数据集上整理交互式工具使用轨迹，对标注数据应用模仿学习，并提出输出空间整形以进一步优化模型的推理行为。实验表明，ToRA模型在10个数学推理数据集上全面超越开源模型，平均绝对提升达13%-19%。值得注意的是，ToRA-7B在竞赛级数据集MATH上达到44.6%准确率，超越最佳开源模型WizardMath-70B达22%绝对值。ToRA-Code-34B成为首个在MATH上准确率突破50%的开源模型，显著超越GPT-4的思维链推理结果，与GPT-4的程序解题性能相当。此外，我们对工具交互在数学推理中的优势与现存挑战进行了全面分析，为未来研究提供了宝贵见解。