Solving tabular math word problems (TMWPs) has become a critical role in evaluating the mathematical reasoning ability of large language models (LLMs), where large-scale TMWP samples are commonly required for LLM fine-tuning. Since the collection of high-quality TMWP datasets is costly and time-consuming, recent research has concentrated on automatic TMWP generation. However, current generated samples usually suffer from issues of either correctness or diversity. In this paper, we propose a Template-driven LLM-paraphrased (TeLL) framework for generating high-quality TMWP samples with diverse backgrounds and accurate tables, questions, answers, and solutions. To this end, we first extract templates from existing real samples to generate initial problems, ensuring correctness. Then, we adopt an LLM to extend templates and paraphrase problems, obtaining diverse TMWP samples. Furthermore, we find the reasoning annotation is important for solving TMWPs. Therefore, we propose to enrich each solution with illustrative reasoning steps. Through the proposed framework, we construct a high-quality dataset TabMWP-TeLL by adhering to the question types in the TabMWP dataset, and we conduct extensive experiments on a variety of LLMs to demonstrate the effectiveness of TabMWP-TeLL in improving TMWP solving performance. The code and data of this paper are available at: https://github.com/Jason8Kang/TELL.
翻译:解决表格数学应用题(TMWP)已成为评估大语言模型(LLM)数学推理能力的关键环节,而LLM微调通常需要大规模TMWP样本。由于高质量TMWP数据集的收集成本高昂且耗时,近期研究集中于自动生成TMWP。然而,现有生成样本常存在正确性或多样性不足的问题。本文提出一种基于模板驱动与LLM复述(TeLL)的框架,用于生成具有多样化背景、精确表格、问题、答案及解法的高质量TMWP样本。为此,我们首先从现有真实样本中提取模板以生成初始问题,确保正确性;随后采用LLM扩展模板并复述问题,从而获得多样化的TMWP样本。此外,我们发现推理标注对解决TMWP至关重要,因此提出通过添加说明性推理步骤来丰富每个解法。基于该框架,我们遵循TabMWP数据集的问题类型构建了高质量数据集TabMWP-TeLL,并在多种LLM上开展大量实验,证明了TabMWP-TeLL在提升TMWP求解性能方面的有效性。本文代码与数据公开于:https://github.com/Jason8Kang/TELL。