Immediate feedback has been shown to improve student learning. In programming courses, immediate, automated feedback is typically provided in the form of pre-defined test cases run by a submission platform. While these are excellent for highlighting the presence of logical errors, they do not provide novice programmers enough scaffolding to help them identify where an error is or how to fix it. To address this, several tools have been developed that provide richer feedback in the form of program repairs. Studies of such tools, however, tend to focus more on whether correct repairs can be generated, rather than how novices are using them. In this paper, we describe our experience of using CLARA, an automated repair tool, to provide feedback to novices. First, we extended CLARA to support a larger subset of the Python language, before integrating it with the Jupyter Notebooks used for our programming exercises. Second, we devised a preliminary study in which students tackled programming problems with and without support of the tool using the 'think aloud' protocol. We found that novices often struggled to understand the proposed repairs, echoing the well-known challenge to understand compiler/interpreter messages. Furthermore, we found that students valued being told where a fix was needed - without necessarily the fix itself - suggesting that 'less may be more' from a pedagogical perspective.
翻译:即时反馈已被证明能促进学生学习。在编程课程中,即时自动化反馈通常以提交平台运行的预定义测试用例形式提供。尽管这些测试用例能有效揭示逻辑错误的存在,但未能为新手程序员提供足够的脚手架支持,帮助他们定位错误或掌握修复方法。为此,研究人员开发了多种工具,通过程序修复形式提供更丰富的反馈。然而,此类工具的研究往往更关注能否生成正确的修复方案,而非新手如何使用这些工具。本文描述了我们在使用自动化修复工具CLARA为新手提供反馈方面的经验。首先,我们扩展了CLARA以支持更大范围的Python语言子集,随后将其集成至编程练习所使用的Jupyter Notebook平台。其次,我们设计了一项初步研究,要求学生在"出声思维"协议下完成编程问题,对比有无工具支持的情况。研究发现,新手常常难以理解所提议的修复方案,这与编译器/解释器信息难以理解这一公认难题如出一辙。此外,我们发现学生更看重被告知需要修改的位置(而非修改方法本身),这表明从教学视角来看"少即是多"。