Recent advances in large language models have enabled them to reach a level of text generation comparable to that of humans. These models show powerful capabilities across a wide range of content, including news article writing, story generation, and scientific writing. Such capability further narrows the gap between human-authored and machine-generated texts, highlighting the importance of deepfake text detection to avoid potential risks such as fake news propagation and plagiarism. However, previous work has been limited in that they testify methods on testbed of specific domains or certain language models. In practical scenarios, the detector faces texts from various domains or LLMs without knowing their sources. To this end, we build a wild testbed by gathering texts from various human writings and deepfake texts generated by different LLMs. Human annotators are only slightly better than random guessing at identifying machine-generated texts. Empirical results on automatic detection methods further showcase the challenges of deepfake text detection in a wild testbed. In addition, out-of-distribution poses a greater challenge for a detector to be employed in realistic application scenarios. We release our resources at https://github.com/yafuly/DeepfakeTextDetect.

翻译：大型语言模型的最新进展使其文本生成能力已达到与人类相当的水平。这些模型在新闻文章撰写、故事创作和科学写作等多种内容领域展现出强大性能，进一步缩小了人类创作与机器生成文本之间的差距，凸显了深度伪造文本检测的重要性——这种检测对于防范虚假新闻传播、学术剽窃等潜在风险至关重要。然而，现有研究存在局限性：它们仅在特定领域或特定语言模型的测试平台上验证检测方法。在实际场景中，检测器需要面对来自不同领域或不同大语言模型的文本，且无法预知其来源。为此，我们通过收集各类人类创作文本及不同大语言模型生成的深度伪造文本，构建了一个野外测试平台。人类标注者对机器生成文本的识别准确率仅略高于随机猜测。针对自动检测方法的实证结果进一步揭示了在野外测试平台中进行深度伪造文本检测的挑战性。此外，分布外数据对检测器在真实应用场景中的部署构成了更大挑战。我们将相关资源发布在 https://github.com/yafuly/DeepfakeTextDetect。