Geometry mathematics problems pose significant challenges for large language models (LLMs) because they involve visual elements and spatial reasoning. Current methods primarily rely on symbolic character awareness to address these problems. Considering geometry problem solving is a relatively nascent field with limited suitable datasets and currently almost no work on solid geometry problem solving, we collect a geometry question-answer dataset by sourcing geometric data from Chinese high school education websites, referred to as GeoMath. It contains solid geometry questions and answers with accurate reasoning steps as compensation for existing plane geometry datasets. Additionally, we propose a Large Multi-modal Model (LMM) framework named Geo-LLaVA, which incorporates retrieval augmentation with supervised fine-tuning (SFT) in the training stage, called meta-training, and employs in-context learning (ICL) during inference to improve performance. Our fine-tuned model with ICL attains the state-of-the-art performance of 65.25% and 42.36% on selected questions of the GeoQA dataset and GeoMath dataset respectively with proper inference steps. Notably, our model initially endows the ability to solve solid geometry problems and supports the generation of reasonable solid geometry picture descriptions and problem-solving steps. Our research sets the stage for further exploration of LLMs in multi-modal math problem-solving, particularly in geometry math problems.

翻译：几何数学问题因其涉及视觉元素和空间推理，对大型语言模型（LLMs）构成了重大挑战。现有方法主要依赖符号字符感知来处理这些问题。考虑到几何问题求解是一个相对新兴的领域，缺乏合适的数据集，且目前在立体几何问题求解方面几乎未有工作涉及，我们通过从中国高中教育网站收集几何数据，构建了一个几何问答数据集，称为GeoMath。该数据集包含立体几何问题与答案，并配有精确的推理步骤，以弥补现有平面几何数据集的不足。此外，我们提出了一种名为Geo-LLaVA的大型多模态模型（LMM）框架，其在训练阶段（称为元训练）结合了检索增强与监督微调（SFT），并在推理阶段采用上下文学习（ICL）以提升性能。我们经过微调并采用ICL的模型，在适当的推理步骤下，在GeoQA数据集和GeoMath数据集的选定问题上分别达到了65.25%和42.36%的最先进性能。值得注意的是，我们的模型初步具备了解决立体几何问题的能力，并支持生成合理的立体几何图像描述和解题步骤。我们的研究为进一步探索LLMs在多模态数学问题求解，特别是几何数学问题中的应用奠定了基础。