Scaling high-quality tutoring is a major challenge in education. Because of the growing demand, many platforms employ novice tutors who, unlike professional educators, struggle to effectively address student mistakes and thus fail to seize prime learning opportunities for students. In this paper, we explore the potential for large language models (LLMs) to assist math tutors in remediating student mistakes. We present ReMath, a benchmark co-developed with experienced math teachers that deconstructs their thought process for remediation. The benchmark consists of three step-by-step tasks: (1) infer the type of student error, (2) determine the strategy to address the error, and (3) generate a response that incorporates that information. We evaluate the performance of state-of-the-art instruct-tuned and dialog models on ReMath. Our findings suggest that although models consistently improve upon original tutor responses, we cannot rely on models alone to remediate mistakes. Providing models with the error type (e.g., the student is guessing) and strategy (e.g., simplify the problem) leads to a 75% improvement in the response quality over models without that information. Nonetheless, despite the improvement, the quality of the best model's responses still falls short of experienced math teachers. Our work sheds light on the potential and limitations of using current LLMs to provide high-quality learning experiences for both tutors and students at scale. Our work is open-sourced at this link: \url{https://github.com/rosewang2008/remath}.

翻译：高质量辅导的规模化扩展是教育领域的一大挑战。由于需求不断增加，许多平台聘请新手导师，但这类导师与专业教育工作者不同，难以有效应对学生错误，从而错失了为学生创造关键学习机会的契机。本文探索了利用大语言模型辅助数学导师纠正学生错误的潜力。我们提出了ReMath基准，与经验丰富的数学教师共同开发，解构了其纠正思路。该基准包含三个逐步任务：（1）推断学生错误类型；（2）确定应对错误的策略；（3）生成包含上述信息的回应。我们评估了最新指令微调模型和对话模型在ReMath上的表现。结果表明，尽管模型在原始导师回应基础上持续改进，但仅靠模型无法完全纠正错误。为模型提供错误类型（如学生在猜测）和策略（如简化问题）后，回应质量相较于无此信息的模型提升了75%。然而，即便有所改进，最佳模型回应的质量仍不及经验丰富的数学教师。本研究揭示了当前利用大语言模型为导师和学生大规模提供高质量学习体验的潜力与局限性。本工作已开源，链接为：\url{https://github.com/rosewang2008/remath}。