Source Code Plagiarism Detection (SCPD) plays an important role in maintaining fairness and academic integrity in software engineering education. Code Evaluation Metrics (CEMs) are developed for assessing code generation tasks. However, it remains unclear whether such metrics can reliably detect plagiarism across different levels of modification (L1-L6), increasing in complexity. In this paper, we perform a comparative empirical study using two open-source labelled datasets, ConPlag (raw and template-free versions) and IRPlag. We evaluate five CEMs, namely CodeBLEU, CrystalBLEU, RUBY, Tree Structured Edit Distance (TSED), and CodeBERTScore. The performance is evaluated using threshold-free ranking-based measures to assess overall, per dataset, and per-level plagiarism performance. The results are compared against state-of-the-art (SOTA) Source Code Plagiarism Detection Tools (SCPDTs), JPlag and Dolos. Our findings show that without preprocessing, Dolos achieves the highest overall ranking performance, while among the individual metrics, CrystalBLEU, CodeBLEU, and RUBY outperform JPlag. Performance is strongest at L1 and drops from L4 onward, while CrystalBLEU remains competitive on L6. With preprocessing, CrystalBLEU surpasses Dolos overall. Per dataset, Dolos achieved the best ranking on the ConPlag raw dataset, while CrystalBLEU was the best-performing metric on the remaining datasets. At the plagiarism levels, Dolos remains strongest on L4, while Crystal-BLEU leads most of the remaining difficult levels. These results indicate that CEMs are comparable to dedicated tools in terms of ranking metrics.

翻译：[translated abstract in Chinese] 源代码剽窃检测（SCPD）在软件工程教育中维护公平性和学术诚信方面发挥着重要作用。代码评估指标（CEMs）是为评估代码生成任务而开发的。然而，尚不清楚这些指标是否能够可靠地检测不同修改级别（L1-L6，复杂度递增）下的剽窃行为。本文利用两个开源带标签数据集ConPlag（原始版本和无模板版本）与IRPlag，进行了实证对比研究。我们评估了五种CEMs：CodeBLEU、CrystalBLEU、RUBY、树结构化编辑距离（TSED）和CodeBERTScore。采用无阈值的排序度量来评估整体性能、各数据集性能以及各层级剽窃性能。结果与最先进的源代码剽窃检测工具（SCPDTs）JPlag和Dolos进行了对比。研究发现，未经预处理时，Dolos的整体排序性能最高；而在单一指标中，CrystalBLEU、CodeBLEU和RUBY优于JPlag。性能在L1级别最强，从L4开始下降，而CrystalBLEU在L6级别仍保持竞争力。经过预处理后，CrystalBLEU的整体性能超越了Dolos。在各数据集上，Dolos在ConPlag原始数据集上排序最佳，而CrystalBLEU在其余数据集上表现最优。在各剽窃层级中，Dolos在L4级别仍最强，而CrystalBLEU在大多数其余高难度级别中领先。这些结果表明，排序度量方面CEMs可与专用工具相媲美。