We introduce MAmmoTH, a series of open-source large language models (LLMs) specifically tailored for general math problem-solving. The MAmmoTH models are trained on MathInstruct, our meticulously curated instruction tuning dataset. MathInstruct is compiled from 13 math datasets with intermediate rationales, six of which have rationales newly curated by us. It presents a unique hybrid of chain-of-thought (CoT) and program-of-thought (PoT) rationales, and also ensures extensive coverage of diverse fields in math. The hybrid of CoT and PoT not only unleashes the potential of tool use but also allows different thought processes for different math problems. As a result, the MAmmoTH series substantially outperform existing open-source models on nine mathematical reasoning datasets across all scales with an average accuracy gain between 16% and 32%. Remarkably, our MAmmoTH-7B model reaches 33% on MATH (a competition-level dataset), which exceeds the best open-source 7B model (WizardMath) by 23%, and the MAmmoTH-34B model achieves 44% accuracy on MATH, even surpassing GPT-4's CoT result. Our work underscores the importance of diverse problem coverage and the use of hybrid rationales in developing superior math generalist models.

翻译：我们提出MAmmoTH系列开源大语言模型（LLMs），专门针对通用数学问题求解任务设计。MAmmoTH模型基于MathInstruct训练——这是经过精心策划的指令调优数据集。MathInstruct整合了13个含中间推理过程的数学数据集，其中六个数据集的推理过程由我们全新构建。该数据集创新性地融合了链式思维（CoT）与程序式思维（PoT）两种推理模式，同时确保覆盖数学领域的多元分支。CoT与PoT的混合不仅释放了工具使用的潜力，还能对不同数学问题启用差异化的思维路径。实验表明，MAmmoTH系列模型在九个数学推理数据集上全面超越现有开源模型，所有规模的平均准确率提升幅度达16%至32%。值得注意的是，我们的MAmmoTH-7B模型在竞赛级数据集MATH上达到33%准确率，较最优开源7B模型（WizardMath）高出23%；而MAmmoTH-34B模型在MATH上实现44%准确率，甚至超越GPT-4的CoT结果。本工作凸显了问题覆盖广度与混合推理模式在构建卓越数学通才模型中的关键价值。