The ease of access to large language models (LLMs) has enabled a widespread of machine-generated texts, and now it is often hard to tell whether a piece of text was human-written or machine-generated. This raises concerns about potential misuse, particularly within educational and academic domains. Thus, it is important to develop practical systems that can automate the process. Here, we present one such system, LLM-DetectAIve, designed for fine-grained detection. Unlike most previous work on machine-generated text detection, which focused on binary classification, LLM-DetectAIve supports four categories: (i) human-written, (ii) machine-generated, (iii) machine-written, then machine-humanized, and (iv) human-written, then machine-polished. Category (iii) aims to detect attempts to obfuscate the fact that a text was machine-generated, while category (iv) looks for cases where the LLM was used to polish a human-written text, which is typically acceptable in academic writing, but not in education. Our experiments show that LLM-DetectAIve can effectively identify the above four categories, which makes it a potentially useful tool in education, academia, and other domains. LLM-DetectAIve is publicly accessible at https://github.com/mbzuai-nlp/LLM-DetectAIve. The video describing our system is available at https://youtu.be/E8eT_bE7k8c.
翻译:大型语言模型(LLM)的易于访问性导致了机器生成文本的广泛传播,如今通常难以判断一段文本是由人类撰写还是机器生成。这引发了对其潜在滥用的担忧,尤其是在教育和学术领域。因此,开发能够自动化此过程的实用系统至关重要。本文介绍了一个这样的系统——LLM-DetectAIve,专为细粒度检测而设计。与以往大多数专注于二元分类的机器生成文本检测工作不同,LLM-DetectAIve支持四个类别:(i)人类撰写,(ii)机器生成,(iii)机器撰写后经人工润色,以及(iv)人类撰写后经机器润色。类别(iii)旨在检测试图掩盖文本为机器生成事实的行为,而类别(iv)则用于识别使用LLM润色人类撰写文本的情况,这在学术写作中通常可以接受,但在教育领域则不然。我们的实验表明,LLM-DetectAIve能够有效识别上述四个类别,这使其成为教育、学术及其他领域中一个潜在有用的工具。LLM-DetectAIve可在 https://github.com/mbzuai-nlp/LLM-DetectAIve 公开访问。描述我们系统的视频可在 https://youtu.be/E8eT_bE7k8c 观看。