Large language models have made significant progress in various language tasks, yet they still struggle with complex mathematics. In this paper, we propose ToRA a series of Tool-integrated Reasoning Agents designed to solve challenging mathematical problems by seamlessly integrating natural language reasoning with the utilization of external tools (e.g., computation libraries and symbolic solvers), thereby amalgamating the analytical prowess of language and the computational efficiency of tools. To train ToRA, we curate interactive tool-use trajectories on mathematical datasets, apply imitation learning on the annotations, and propose output space shaping to further refine models' reasoning behavior. As a result, ToRA models significantly outperform open-source models on 10 mathematical reasoning datasets across all scales with 13%-19% absolute improvements on average. Notably, ToRA-7B reaches 44.6% on the competition-level dataset MATH, surpassing the best open-source model WizardMath-70B by 22% absolute. ToRA-Code-34B is also the first open-source model that achieves an accuracy exceeding 50% on MATH, which significantly outperforms GPT-4's CoT result, and is competitive with GPT-4 solving problems with programs. Additionally, we conduct a comprehensive analysis of the benefits and remaining challenges of tool interaction for mathematical reasoning, providing valuable insights for future research.

翻译：大语言模型在各种语言任务上取得了显著进展，但在复杂数学问题上仍表现不佳。本文提出ToRA系列工具集成推理智能体，通过将自然语言推理与外部工具（如计算库和符号求解器）无缝结合，融合语言的分析能力与工具的计算效率，以解决具有挑战性的数学问题。为训练ToRA，我们在数学数据集中整理交互式工具使用轨迹，对标注数据应用模仿学习，并提出输出空间整形方法以进一步优化模型的推理行为。实验表明，ToRA模型在10个数学推理数据集上全面超越开源模型，平均绝对提升达13%-19%。特别地，ToRA-7B在竞赛级数据集MATH上达到44.6%的准确率，比最优开源模型WizardMath-70B绝对提升22%。ToRA-Code-34B是首个在MATH上准确率突破50%的开源模型，显著超越GPT-4的思维链结果，并与使用程序求解问题的GPT-4性能相当。此外，我们全面分析了工具交互对数学推理的益处与现存挑战，为未来研究提供了重要启示。