To combat the potential misuse of Natural Language Generation (NLG) technology, a variety of algorithms have been developed for the detection of AI-generated texts. Traditionally, this task is treated as a binary classification problem. Although supervised learning has demonstrated promising results, acquiring labeled data for detection purposes poses real-world challenges and the risk of overfitting. In an effort to address these issues, we delve into the realm of zero-shot machine-generated text detection. Existing zero-shot detectors, typically designed for specific tasks or topics, often assume uniform testing scenarios, limiting their practicality. In our research, we explore various advanced Large Language Models (LLMs) and their specialized variants, contributing to this field in several ways. In empirical studies, we uncover a significant correlation between topics and detection performance. Secondly, we delve into the influence of topic shifts on zero-shot detectors. These investigations shed light on the adaptability and robustness of these detection methods across diverse topics.
翻译:为应对自然语言生成(NLG)技术被滥用的潜在风险,目前已开发出多种检测人工智能生成文本的算法。传统上,该任务被视为二分类问题。尽管监督学习已展现出可喜成果,但为检测任务获取标注数据面临现实挑战且存在过拟合风险。为应对这些问题,我们深入探索了零样本机器生成文本检测领域。现有零样本检测器通常针对特定任务或主题设计,常假设测试场景具有均一性,限制了其实用性。在本研究中,我们探索了多种先进的大语言模型(LLM)及其专业变体,从多个维度为该领域做出贡献。在实证研究中,我们揭示了主题与检测性能之间的显著相关性。其次,我们深入研究了主题偏移对零样本检测器的影响。这些研究揭示了这些检测方法在不同主题上的适应性和鲁棒性。