In this work, we developed an algorithm for detecting code quality issues in the templates of online programming tasks, validated it, and conducted an empirical study on the dataset of student solutions. The algorithm consists of analyzing recurring unfixed issues in solutions of different students, matching them with the code of the template, and then filtering the results. Our manual validation on a subset of tasks demonstrated a precision of 80.8% and a recall of 73.3%. We used the algorithm on 415 Java tasks from the JetBrains Academy platform and discovered that as much as 14.7% of tasks have at least one issue in their template, thus making it harder for students to learn good code quality practices. We describe our results in detail, provide several motivating examples and specific cases, and share the feedback of the developers of the platform, who fixed 51 issues based on the output of our approach.
翻译:本文提出了一种用于检测在线编程任务模板中代码质量问题的算法,对其进行了验证,并基于学生解决方案数据集开展了实证研究。该算法包括分析不同学生解决方案中反复出现的未修复问题、将其与模板代码进行匹配,最后对结果进行筛选。我们通过对任务子集进行人工验证,表明该算法的精确率达80.8%,召回率达73.3%。我们将该算法应用于JetBrains Academy平台的415个Java任务,发现多达14.7%的任务模板存在至少一个问题,从而导致学生难以掌握良好的代码质量实践。我们详细描述了研究结果,提供了若干激励性示例和具体案例,并分享了该平台开发人员的反馈,他们根据我们方法输出的结果修复了51个问题。