To combat the potential misuse of Natural Language Generation (NLG) technology, a variety of algorithms have been developed for the detection of AI-generated texts. Traditionally, this task is treated as a binary classification problem. Although supervised learning has demonstrated promising results, acquiring labeled data for detection purposes poses real-world challenges and the risk of overfitting. In an effort to address these issues, we delve into the realm of zero-shot machine-generated text detection. Existing zero-shot detectors, typically designed for specific tasks or topics, often assume uniform testing scenarios, limiting their practicality. In our research, we explore various advanced Large Language Models (LLMs) and their specialized variants, contributing to this field in several ways. In empirical studies, we uncover a significant correlation between topics and detection performance. Secondly, we delve into the influence of topic shifts on zero-shot detectors. These investigations shed light on the adaptability and robustness of these detection methods across diverse topics. The code is available at \url{https://github.com/yfzhang114/robustness-detection}.
翻译:为应对自然语言生成(NLG)技术潜在的滥用风险,学界已开发多种算法用于检测AI生成的文本。传统上,该任务被视作二元分类问题。尽管监督学习展现出可观成效,但为检测任务获取标注数据不仅面临现实挑战,更存在过拟合风险。针对这些问题,我们深入探索了零样本机器生成文本检测领域。现有零样本检测器通常针对特定任务或主题设计,往往假设统一测试场景,这限制了其实用性。在本研究中,我们考察了多种先进大语言模型(LLMs)及其专业变体,从多个维度为该领域做出贡献。通过实证研究,我们发现了主题与检测性能之间的显著相关性。其次,我们深入探究了主题偏移对零样本检测器的影响。这些研究揭示了不同主题下检测方法的适应性与鲁棒性。相关代码开源于 \url{https://github.com/yfzhang114/robustness-detection}。