With the prevalence of imperfect but capable LLMs in software development, it becomes increasingly important to develop debugging skills -- to form hypotheses about the source of error in both human codes and codes from their AI pair programmers. Despite the necessity, hypothesis construction in debugging is rarely taught. In this work, we explore whether LLMs can be used to train novices on hypothesis construction, by designing a theoretically motivated, LLM-augmented tutor -- HypoCompass. HypoCompass uses LLMs to generate rich training materials guided by learning principles and presents them in a learning-by-teaching environment, where LLMs act as students who write bugs, and human novices play the role of Teaching Assistants to help debug and fix the code. Evaluations show that \sysname makes high-quality training materials four times more efficiently than humans and brings significant learning gain: 19 novices improved their pre-to-post test performances by 12%, with a reduced completion time of 14%.
翻译:随着不完善但功能强大的LLM在软件开发中的普及,培养调试技能(即对人类代码及其AI编程搭档代码中的错误来源形成假设)变得日益重要。尽管必要性突出,但调试中的假设构建却鲜有教学。在本研究中,我们探讨如何利用LLM训练初学者进行假设构建,通过设计一个理论驱动、LLM增强的导师系统——HypoCompass。该系统借助LLM依据学习原则生成丰富的训练材料,并在“教学相长”环境中呈现,其中LLM扮演编写错误代码的学生角色,而人类初学者则充当教学助理,协助调试与修复代码。评估显示,HypoCompass生成高质量训练材料的效率比人类高出四倍,并带来显著的学习增益:19名初学者的前后测成绩提升12%,完成时间缩短14%。