Modeling plausible student misconceptions is critical for AI in education. In this work, we examine how large language models (LLMs) reason about misconceptions when generating multiple-choice distractors, a task that requires modeling incorrect yet plausible answers by coordinating solution knowledge, simulating student misconceptions, and evaluating plausibility. We introduce a taxonomy for analyzing the strategies used by state-of-the-art LLMs, examining their reasoning procedures and comparing them to established best practices in the learning sciences. Our structured analysis reveals a surprising alignment between their processes and best practices: the models typically solve the problem correctly first, then articulate and simulate multiple potential misconceptions, and finally select a set of distractors. An analysis of failure modes reveals that errors arise primarily from failures in recovering the correct solution and selecting among response candidates, rather than simulating errors or structuring the process. Consistent with these results, we find that providing the correct solution in the prompt improves alignment with human-authored distractors by 8%, highlighting the critical role of anchoring to the correct solution when generating plausible incorrect student reasoning. Overall, our analysis offers a structured and interpretable lens into LLMs' ability to model incorrect student reasoning and produce high-quality distractors.

翻译：在教育人工智能领域，准确模拟学生可能存在的误解至关重要。本研究探讨了大型语言模型（LLMs）在生成多项选择题干扰项时如何对错误概念进行推理——这项任务要求模型通过协调解题知识、模拟学生误解并评估合理性来构建错误但看似合理的答案。我们提出了一种分类体系，用于分析最先进LLMs所采用的策略，检验其推理过程，并将其与学习科学中既定的最佳实践进行比较。我们的结构化分析揭示了模型推理过程与最佳实践之间存在惊人的一致性：模型通常先正确解决问题，然后阐明并模拟多种潜在误解，最后筛选出一组干扰项。通过对失败模式的分析发现，错误主要源于正确解题过程的失效和候选答案的选择失误，而非错误模拟或流程结构问题。与这些结果一致的是，我们发现若在提示中提供正确答案，可使模型生成的干扰项与人工编写干扰项的一致性提升8%，这凸显了在生成合理的学生错误推理时锚定正确答案的关键作用。总体而言，我们的分析为理解LLMs模拟学生错误推理及生成高质量干扰项的能力提供了一个结构化且可解释的研究视角。