In recent years, there has been remarkable progress in leveraging Language Models (LMs), encompassing Pre-trained Language Models (PLMs) and Large-scale Language Models (LLMs), within the domain of mathematics. This paper conducts a comprehensive survey of mathematical LMs, systematically categorizing pivotal research endeavors from two distinct perspectives: tasks and methodologies. The landscape reveals a large number of proposed mathematical LLMs, which are further delineated into instruction learning, tool-based methods, fundamental CoT techniques, and advanced CoT methodologies. In addition, our survey entails the compilation of over 60 mathematical datasets, including training datasets, benchmark datasets, and augmented datasets. Addressing the primary challenges and delineating future trajectories within the field of mathematical LMs, this survey is positioned as a valuable resource, poised to facilitate and inspire future innovation among researchers invested in advancing this domain.

翻译：近年来，语言模型（LMs），包括预训练语言模型（PLMs）和大规模语言模型（LLMs）在数学领域取得了显著进展。本文对数学语言模型进行了全面综述，从任务和方法两个不同维度系统性地分类了关键研究成果。研究格局显示出大量已提出的数学LLM方案，这些方案进一步被细分为指令学习、基于工具的方法、基础CoT技术以及高级CoT方法论。此外，本综述还整理了超过60个数学数据集，包括训练数据集、基准数据集和增强数据集。针对数学语言模型领域的主要挑战和未来发展方向，本文提供了有价值的参考资源，有助于促进和启发致力于该领域研究者的创新工作。