Widely applied large language models (LLMs) can generate human-like content, raising concerns about the abuse of LLMs. Therefore, it is important to build strong AI-generated text (AIGT) detectors. Current works only consider document-level AIGT detection, therefore, in this paper, we first introduce a sentence-level detection challenge by synthesizing a dataset that contains documents that are polished with LLMs, that is, the documents contain sentences written by humans and sentences modified by LLMs. Then we propose \textbf{Seq}uence \textbf{X} (Check) \textbf{GPT}, a novel method that utilizes log probability lists from white-box LLMs as features for sentence-level AIGT detection. These features are composed like \textit{waves} in speech processing and cannot be studied by LLMs. Therefore, we build SeqXGPT based on convolution and self-attention networks. We test it in both sentence and document-level detection challenges. Experimental results show that previous methods struggle in solving sentence-level AIGT detection, while our method not only significantly surpasses baseline methods in both sentence and document-level detection challenges but also exhibits strong generalization capabilities.
翻译:广泛使用的大型语言模型(LLM)能够生成类人内容,引发了对LLM滥用的担忧。因此,构建强大的AI生成文本(AIGT)检测器至关重要。现有工作仅考虑文档级别的AIGT检测,本文首次通过合成一个包含经LLM润色文档的数据集(即文档中既包含人类撰写的句子,也包含由LLM修改的句子),引入了句子级别的检测挑战。随后,我们提出序列X(检查)GPT(SeqXGPT),一种利用白盒LLM的对数概率列表作为句子级AIGT检测特征的新方法。这些特征在语音处理中如同“波形”般组成,无法被LLM自身学习,因此我们基于卷积和自注意力网络构建了SeqXGPT。我们在句子和文档级别的检测挑战中均对其进行了测试。实验结果表明,以往方法在解决句子级AIGT检测时存在困难,而我们的方法不仅在句子和文档级检测挑战中均显著优于基线方法,还展现出强大的泛化能力。