Large language models (LLMs) have achieved human-level text generation, emphasizing the need for effective AI-generated text detection to mitigate risks like the spread of fake news and plagiarism. Existing research has been constrained by evaluating detection methods on specific domains or particular language models. In practical scenarios, however, the detector faces texts from various domains or LLMs without knowing their sources. To this end, we build a comprehensive testbed by gathering texts from diverse human writings and texts generated by different LLMs. Empirical results show challenges in distinguishing machine-generated texts from human-authored ones across various scenarios, especially out-of-distribution. These challenges are due to the decreasing linguistic distinctions between the two sources. Despite challenges, the top-performing detector can identify 86.54% out-of-domain texts generated by a new LLM, indicating the feasibility for application scenarios. We release our resources at https://github.com/yafuly/MAGE.

翻译：大型语言模型(LLMs)已实现人类级别的文本生成能力，这凸显了对高效AI生成文本检测方法的需求，以降低假新闻传播和学术剽窃等风险。现有研究受限于在特定领域或特定语言模型上评估检测方法。然而在实际场景中，检测器需要面对来自不同领域或不同LLMs的文本，且无法获知其来源。为此，我们通过收集人类撰写的多样化文本以及不同LLMs生成的文本，构建了综合性测试平台。实验结果表明，在跨场景（尤其是数据分布外场景）下区分机器生成文本与人类撰写文本仍存在挑战，其根源在于两类文本的语言差异持续缩小。尽管面临挑战，最优检测器仍能识别86.54%由新LLM生成的域外文本，这表明该方法在应用场景中具有可行性。我们已在https://github.com/yafuly/MAGE 公开相关资源。