Students often struggle with solving programming problems when learning to code, especially when they have to do it online, with one of the most common disadvantages of working online being the lack of personalized help. This help can be provided as next-step hint generation, i.e., showing a student what specific small step they need to do next to get to the correct solution. There are many ways to generate such hints, with large language models (LLMs) being among the most actively studied right now. While LLMs constitute a promising technology for providing personalized help, combining them with other techniques, such as static analysis, can significantly improve the output quality. In this work, we utilize this idea and propose a novel system to provide both textual and code hints for programming tasks. The pipeline of the proposed approach uses a chain-of-thought prompting technique and consists of three distinct steps: (1) generating subgoals - a list of actions to proceed with the task from the current student's solution, (2) generating the code to achieve the next subgoal, and (3) generating the text to describe this needed action. During the second step, we apply static analysis to the generated code to control its size and quality. The tool is implemented as a modification to the open-source JetBrains Academy plugin, supporting students in their in-IDE courses. To evaluate our approach, we propose a list of criteria for all steps in our pipeline and conduct two rounds of expert validation. Finally, we evaluate the next-step hints in a classroom with 14 students from two universities. Our results show that both forms of the hints - textual and code - were helpful for the students, and the proposed system helped them to proceed with the coding tasks.
翻译:学生在学习编程时,常因在线解决编程问题而陷入困境,其中在线学习最常见的劣势之一便是缺乏个性化指导。这种指导可以以"下一步提示生成"的形式提供,即向学生展示接下来需要执行的具体小步骤以达成正确解。生成此类提示的方法众多,大型语言模型(LLMs)是当前最受关注的技术之一。尽管LLMs为提供个性化指导带来了希望,但将其与静态分析等其他技术结合,能显著提升输出质量。本研究基于这一理念,提出了一种为编程任务同时提供文本提示与代码提示的新颖系统。该方法的流程采用思维链提示技术,包含三个独立步骤:(1) 生成子目标——从学生当前解决方案出发,列出推进任务所需执行的操作序列;(2) 生成实现下一个子目标的代码;(3) 生成描述该必要操作的文本。在第二步中,我们对生成的代码应用静态分析以控制其规模与质量。该工具通过修改开源JetBrains Academy插件实现,支持学生在IDE内课程中的学习。为评估本方法,我们针对流程各步骤提出了一套标准,并进行了两轮专家验证。最后,我们在包含两所大学14名学生的课堂中评估了下一步提示。结果表明:文本与代码两种形式的提示均对学生有帮助,所提出的系统能有效协助他们推进编程任务。