Solving math story problems is a complex task for students and NLP models alike, requiring them to understand the world as described in the story and reason over it to compute an answer. Recent years have seen impressive performance on automatically solving these problems with large pre-trained language models and innovative techniques to prompt them. However, it remains unclear if these models possess accurate representations of mathematical concepts. This leads to lack of interpretability and trustworthiness which impedes their usefulness in various applications. In this paper, we consolidate previous work on categorizing and representing math story problems and develop MathWorld, which is a graph-based semantic formalism specific for the domain of math story problems. With MathWorld, we can assign world models to math story problems which represent the situations and actions introduced in the text and their mathematical relationships. We combine math story problems from several existing datasets and annotate a corpus of 1,019 problems and 3,204 logical forms with MathWorld. Using this data, we demonstrate the following use cases of MathWorld: (1) prompting language models with synthetically generated question-answer pairs to probe their reasoning and world modeling abilities, and (2) generating new problems by using the world models as a design space.

翻译：解决数学故事题对学生和自然语言处理模型而言都是一项复杂任务，要求他们理解故事中描述的世界，并在此基础上进行推理以计算出答案。近年来，利用大型预训练语言模型和创新的提示技术自动求解此类问题取得了令人瞩目的成果。然而，这些模型是否具备准确的数学概念表征仍不明确。这导致模型缺乏可解释性和可信度，从而限制了其在各种应用中的实用性。本文整合了先前关于数学故事题分类与表征的研究工作，开发了MathWorld——一种专为数学故事题领域设计的基于图的语义形式化系统。借助MathWorld，我们可为数学故事题分配世界模型，以表征文本中引入的情境、动作及其数学关系。我们整合了多个现有数据集中的数学故事题，并使用MathWorld标注了包含1,019道题目和3,204个逻辑形式的语料库。基于该数据，我们展示了MathWorld的以下用途：(1) 使用合成生成的问答对提示语言模型，探究其推理与世界建模能力；(2) 将世界模型作为设计空间，生成新问题。