In many MOOCs, whenever a student completes a programming task, they can see previous solutions of other students to find potentially different ways of solving the problem and learn new coding constructs. However, a lot of MOOCs simply show the most recent solutions, disregarding their diversity or quality. To solve this novel problem, we adapted the existing plagiarism detection tool JPlag to Python submissions on Hyperskill, a popular MOOC platform. However, due to the tool's inner algorithm, it fully processed only 46 out of 867 studied tasks. Therefore, we developed our own tool called Rhubarb. This tool first standardizes solutions that are algorithmically the same, then calculates the structure-aware edit distance between them, and then applies clustering. Finally, it selects one example from each of the largest clusters, taking into account their code quality. Rhubarb was able to handle all 867 tasks successfully. We compared approaches on a set of 59 tasks that both tools could process. Eight experts rated the selected solutions based on diversity, code quality, and usefulness. The default platform approach of selecting recent submissions received on average 3.12 out of 5, JPlag - 3.77, Rhubarb - 3.50. Since in the real MOOC, it is imperative to process everything, we created a system that uses JPlag on the 5.3% of tasks it fully processes and Rhubarb on the remaining 94.7%.
翻译:在许多大规模开放在线课程(MOOC)中,每当学生完成编程任务后,他们可以查看其他学生的历史解决方案,以发现潜在的不同解题思路并学习新的编码结构。然而,大量MOOC仅简单展示最近的解决方案,忽视了其多样性与质量。为解决这一全新问题,我们针对主流MOOC平台Hyperskill上的Python提交作业,改造了现有抄袭检测工具JPlag。但由于该工具内部算法的限制,在867个研究任务中仅能完整处理46个。为此,我们自主研发了名为Rhubarb的工具。该工具首先对算法相同的解决方案进行标准化处理,随后计算它们之间的结构感知编辑距离,再实施聚类分析。最后从最大的聚类中各选取一个示例,同时兼顾代码质量。Rhubarb成功处理了全部867个任务。我们针对两个工具均可处理的59个任务进行了方法对比。八位专家根据多样性、代码质量和实用性对筛选出的解决方案进行评分。默认平台按最新提交选择的方案平均得分为3.12(满分5分),JPlag为3.77,Rhubarb为3.50。鉴于真实MOOC必须处理全部任务,我们构建了一个系统:对于能完整处理的5.3%任务使用JPlag,其余94.7%任务采用Rhubarb。