Large Language Models (LLMs) have demonstrated remarkable proficiency in generating code. However, the misuse of LLM-generated (synthetic) code has raised concerns in both educational and industrial contexts, underscoring the urgent need for synthetic code detectors. Existing methods for detecting synthetic content are primarily designed for general text and struggle with code due to the unique grammatical structure of programming languages and the presence of numerous ''low-entropy'' tokens. Building on this, our work proposes a novel zero-shot synthetic code detector based on the similarity between the original code and its LLM-rewritten variants. Our method is based on the observation that differences between LLM-rewritten and original code tend to be smaller when the original code is synthetic. We utilize self-supervised contrastive learning to train a code similarity model and evaluate our approach on two synthetic code detection benchmarks. Our results demonstrate a significant improvement over existing SOTA synthetic content detectors, with AUROC scores increasing by 20.5% on the APPS benchmark and 29.1% on the MBPP benchmark.

翻译：大型语言模型（LLM）在代码生成方面展现出卓越的能力。然而，LLM生成（合成）代码的滥用已在教育和工业领域引发担忧，凸显了对合成代码检测器的迫切需求。现有的合成内容检测方法主要针对通用文本设计，由于编程语言独特的语法结构以及大量"低熵"标记的存在，这些方法在处理代码时面临困难。基于此，我们的工作提出了一种新颖的零样本合成代码检测器，其核心在于原始代码与其LLM重写变体之间的相似性。我们的方法基于以下观察：当原始代码为合成代码时，LLM重写代码与原始代码之间的差异往往更小。我们利用自监督对比学习训练代码相似性模型，并在两个合成代码检测基准上评估我们的方法。实验结果表明，我们的方法相较于现有SOTA合成内容检测器有显著提升，在APPS基准上AUROC分数提高了20.5%，在MBPP基准上提高了29.1%。