Large language models (LLMs) have been massively applied to many tasks, often surpassing state-of-the-art approaches. While their effectiveness in code generation has been extensively studied (e.g., AlphaCode), their potential for code detection remains unexplored. This work presents the first analysis of code detection using LLMs. Our study examines essential kernels, including matrix multiplication, convolution, and fast-fourier transform, implemented in C/C++. We propose both a preliminary, naive prompt and a novel prompting strategy for code detection. Results reveal that conventional prompting achieves great precision but poor accuracy (68.8%, 22.3%, and 79.2% for GEMM, convolution, and FFT, respectively) due to a high number of false positives. Our novel prompting strategy substantially reduces false positives, resulting in excellent overall accuracy (91.1%, 97.9%, and 99.7%, respectively). These results pose a considerable challenge to existing state-of-the-art code detection methods.

翻译：大型语言模型（LLMs）已被广泛应用于众多任务，且常超越当前最先进的方法。尽管其在代码生成方面的有效性已得到广泛研究（如AlphaCode），但在代码检测领域的潜力尚未被探索。本文首次提出了利用LLMs进行代码检测的分析。我们的研究考察了C/C++实现的关键内核，包括矩阵乘法、卷积和快速傅里叶变换。我们提出了初步的朴素提示方法和一种新颖的代码检测提示策略。结果表明，传统提示方法虽能达到极高精度，但因大量误报导致准确性较低（GEMM、卷积和FFT分别为68.8%、22.3%和79.2%）。我们的新颖提示策略显著降低了误报率，从而实现了极佳的整体准确性（分别为91.1%、97.9%和99.7%）。这些结果对现有的最先进代码检测方法构成了显著挑战。