With the prevalence of imperfect but capable LLMs in software development, it becomes increasingly important for developers to cultivate debugging skills -- to form hypotheses about the source of error in both their own codes and codes produced by their AI pair programmers. Despite the necessity, hypothesis construction in debugging is rarely taught due to a lack of explicit instruction. In this work, we explore whether LLMs can be used to train novices on hypothesis construction, by designing a theoretically motivated, LLM-augmented tutor -- HypoCompass. HypoCompass relies on LLMs for generating rich training materials guided by learning principles and presents them in a learning-by-teaching environment, where LLMs act as students who write bugs and attempt to fix them, and human novices focus on debugging in the role of a Teaching Assistant. Evaluations show that HypoCompass consistently generates high-quality training materials, and brings significant learning gain: In a pre-to-post test setup, 10 novices improved their performances by 17%, with a reduced completion time of 13%.
翻译:随着不完美但具备能力的LLM在软件开发中的普及,开发者培养调试技能(即对自身代码及AI结对程序员生成的代码错误根源提出假设)变得愈发重要。尽管这一需求迫切,但调试中的假设构建由于缺乏明确的教学指导而鲜少被教授。本研究探索如何利用LLM训练新手构建假设,通过设计一个理论驱动、LLM增强的导师系统——HypoCompass。该工具依托LLM根据学习原则生成丰富的训练素材,并在"从教中学"环境中呈现——LLM扮演编写Bug并尝试修复的学生角色,而人类新手则以助教身份专注调试。评估表明,HypoCompass能持续生成高质量训练材料并带来显著学习增益:在前测-后测实验中,10名新手的测试成绩提升17%,完成任务时间缩短13%。