In this paper, we describe and present the first dataset of source code plagiarism specifically aimed at contest plagiarism. The dataset contains 251 pairs of plagiarized solutions of competitive programming tasks in Java, as well as 660 non-plagiarized ones, however, the described approach can be used to extend the dataset in the future. Importantly, each pair comes in two versions: (a) "raw" and (b) with participants' repeated template code removed, allowing for evaluating tools in different settings. We used the collected dataset to compare the available source code plagiarism detection tools, including state-of-the-art ones, specifically in their ability to detect contest plagiarism. Our results indicate that the tools show significantly worse performance on the contest plagiarism because of the template code and the presence of other misleadingly similar code. Of the tested tools, token-based ones demonstrated the best performance in both variants of the dataset.
翻译:本文描述并首次提出了专门针对竞赛抄袭的源代码数据集。该数据集包含251对Java竞赛编程任务的抄袭解决方案,以及660对非抄袭解决方案,但所描述的方法可用于未来扩展数据集。重要的是,每对数据均提供两个版本:(a) "原始"版本和(b) 移除参与者重复模板代码的版本,从而支持在不同场景下评估工具。我们利用该数据集比较了现有源代码抄袭检测工具(包括最先进工具)在竞赛抄袭检测中的能力。结果表明,由于模板代码及其他误导性相似代码的存在,这些工具在竞赛抄袭场景中的性能显著下降。在测试工具中,基于令牌的工具在数据集的两种变体上均表现出最佳性能。