Large language models (LLMs) have shown the ability to produce fluent and cogent content, presenting both productivity opportunities and societal risks. To build trustworthy AI systems, it is imperative to distinguish between machine-generated and human-authored content. The leading zero-shot detector, DetectGPT, showcases commendable performance but is marred by its intensive computational costs. In this paper, we introduce the concept of conditional probability curvature to elucidate discrepancies in word choices between LLMs and humans within a given context. Utilizing this curvature as a foundational metric, we present **Fast-DetectGPT**, an optimized zero-shot detector, which substitutes DetectGPT's perturbation step with a more efficient sampling step. Our evaluations on various datasets, source models, and test conditions indicate that Fast-DetectGPT not only surpasses DetectGPT by a relative around 75% in both the white-box and black-box settings but also accelerates the detection process by a factor of 340, as detailed in Table 1. See \url{https://github.com/baoguangsheng/fast-detect-gpt} for code, data, and results.
翻译:大型语言模型(LLMs)已展现出生成流畅且连贯内容的能力,这既带来了生产力机遇,也引发了社会风险。为构建可信赖的AI系统,区分机器生成内容与人类创作内容势在必行。领先的零样本检测器DetectGPT展现了出色的性能,但其高昂的计算成本成为制约因素。本文提出条件概率曲率概念,用以阐释特定上下文中LLMs与人类在词汇选择上的差异。我们将该曲率作为基础度量指标,提出优化后的零样本检测器**Fast-DetectGPT**,该检测器采用更高效的采样步骤替代DetectGPT的扰动步骤。我们在多个数据集、源模型及测试条件下的评估表明,Fast-DetectGPT在白盒与黑盒设置中不仅较DetectGPT相对提升约75%,更将检测过程加速340倍(详见表1)。代码、数据及结果参见\url{https://github.com/baoguangsheng/fast-detect-gpt}。