Physics problems constitute a significant aspect of reasoning, necessitating complicated reasoning ability and abundant physics knowledge. However, existing large language models (LLMs) frequently fail due to a lack of knowledge or incorrect knowledge application. To mitigate these issues, we propose Physics Reasoner, a knowledge-augmented framework to solve physics problems with LLMs. Specifically, the proposed framework constructs a comprehensive formula set to provide explicit physics knowledge and utilizes checklists containing detailed instructions to guide effective knowledge application. Namely, given a physics problem, Physics Reasoner solves it through three stages: problem analysis, formula retrieval, and guided reasoning. During the process, checklists are employed to enhance LLMs' self-improvement in the analysis and reasoning stages. Empirically, Physics Reasoner mitigates the issues of insufficient knowledge and incorrect application, achieving state-of-the-art performance on SciBench with an average accuracy improvement of 5.8%.
翻译:物理问题构成推理任务的重要方面,需要复杂的推理能力和丰富的物理知识。然而,现有的大语言模型(LLMs)常因知识缺乏或知识应用错误而求解失败。为缓解这些问题,我们提出物理推理器(Physics Reasoner),一种基于知识增强的框架,用于利用大语言模型求解物理问题。具体而言,该框架构建了一个全面的公式集以提供显式的物理知识,并利用包含详细指令的检查清单来指导有效的知识应用。即给定一个物理问题,物理推理器通过三个阶段进行求解:问题分析、公式检索和引导推理。在此过程中,检查清单被用于增强大语言模型在分析和推理阶段的自我改进能力。实验表明,物理推理器有效缓解了知识不足和应用错误的问题,在SciBench基准上取得了最先进的性能,平均准确率提升了5.8%。