Reasoning about code and explaining its purpose are fundamental skills for computer scientists. There has been extensive research in the field of computing education on the relationship between a student's ability to explain code and other skills such as writing and tracing code. In particular, the ability to describe at a high-level of abstraction how code will behave over all possible inputs correlates strongly with code writing skills. However, developing the expertise to comprehend and explain code accurately and succinctly is a challenge for many students. Existing pedagogical approaches that scaffold the ability to explain code, such as producing exemplar code explanations on demand, do not currently scale well to large classrooms. The recent emergence of powerful large language models (LLMs) may offer a solution. In this paper, we explore the potential of LLMs in generating explanations that can serve as examples to scaffold students' ability to understand and explain code. To evaluate LLM-created explanations, we compare them with explanations created by students in a large course ($n \approx 1000$) with respect to accuracy, understandability and length. We find that LLM-created explanations, which can be produced automatically on demand, are rated as being significantly easier to understand and more accurate summaries of code than student-created explanations. We discuss the significance of this finding, and suggest how such models can be incorporated into introductory programming education.
翻译:理解代码并解释其目的是计算机科学家的基本技能。在计算机教育领域,已有大量研究探讨学生解释代码的能力与编写、追踪代码等其他技能之间的关系。具体而言,从高抽象层次描述代码在所有可能输入下的行为方式的能力与代码编写技能呈强相关。然而,对于许多学生而言,培养准确、简洁地理解和解释代码的专业能力仍是一大挑战。现有的教学法(如按需生成示例代码解释)虽然能支持解释代码的能力培养,但难以大规模应用于大型课堂。近年来强大语言模型(LLMs)的兴起可能为此提供解决方案。本文探索了LLMs在生成解释方面的潜力,这些解释可作为示例来帮助学生构建理解与解释代码的能力。为评估LLM生成的解释,我们将其与大型课程($n \approx 1000$)中学生生成的解释进行了准确性、可理解性和长度方面的比较。研究发现,与学生的解释相比,可自动按需生成的LLM解释被评价为显著更易理解且能更准确地概括代码。我们讨论了这一发现的意义,并提出了如何将此类模型融入入门编程教育的建议。