In many MOOCs, whenever a student completes a programming task, they can see previous solutions of other students to find potentially different ways of solving the problem and to learn new coding constructs. However, a lot of MOOCs simply show the most recent solutions, disregarding their diversity or quality, and thus hindering the students' opportunity to learn. In this work, we explore this novel problem for the first time. To solve it, we adapted the existing plagiarism detection tool JPlag to Python submissions on Hyperskill, a popular MOOC platform. However, due to the tool's inner algorithm, JPLag fully processed only 46 out of 867 studied tasks. Therefore, we developed our own tool called Rhubarb. This tool first standardizes solutions that are algorithmically the same, then calculates the structure-aware edit distance between them, and then applies clustering. Finally, it selects one example from each of the largest clusters, thus ensuring their diversity. Rhubarb was able to handle all 867 tasks successfully. We compared different approaches on a set of 59 real-life tasks that both tools could process. Eight experts rated the selected solutions based on diversity, code quality, and usefulness. The default platform approach of simply selecting recent submissions received on average 3.12 out of 5, JPlag - 3.77, Rhubarb - 3.50. To ensure both quality and coverage, we created a system that combines both tools. We conclude our work by discussing the future of this new problem and the research needed to solve it better.
翻译:在许多大规模在线开放课程(MOOC)中,当学生完成编程任务后,可以查看其他学生先前提交的解决方案,以发现潜在的不同解题方法并学习新的编码结构。然而,许多MOOC平台仅展示最近的解决方案,忽略了其多样性与质量,从而限制了学生的学习机会。本研究首次探索这一新颖问题。为解决该问题,我们改编了现有的抄袭检测工具JPlag,使其适用于流行MOOC平台Hyperskill上的Python作业提交。但由于该工具的内部算法限制,JPlag在研究的867个任务中仅完整处理了46个。为此,我们开发了名为Rhubarb的新工具。该工具首先对算法相同的解决方案进行标准化处理,随后计算其间结构感知的编辑距离,并应用聚类分析。最终从每个最大聚类中选取一个示例,从而确保解决方案的多样性。Rhubarb成功处理了全部867个任务。我们在两种工具均可处理的59个真实任务集上比较了不同方法。八位专家根据多样性、代码质量和实用性对选取的解决方案进行评分。平台默认的近期提交选取方法平均得分为3.12(满分5分),JPlag得3.77分,Rhubarb得3.50分。为兼顾质量与覆盖率,我们构建了融合两种工具的系统。最后通过探讨这一新问题的未来发展方向及所需的研究工作来总结本文。