ChatGPT has enabled access to AI-generated writing for the masses, and within just a few months, this product has disrupted the knowledge economy, initiating a culture shift in the way people work, learn, and write. The need to discriminate human writing from AI is now both critical and urgent, particularly in domains like higher education and academic writing, where AI had not been a significant threat or contributor to authorship. Addressing this need, we developed a method for discriminating text generated by ChatGPT from (human) academic scientists, relying on prevalent and accessible supervised classification methods. We focused on how a particular group of humans, academic scientists, write differently than ChatGPT, and this targeted approach led to the discovery of new features for discriminating (these) humans from AI; as examples, scientists write long paragraphs and have a penchant for equivocal language, frequently using words like but, however, and although. With a set of 20 features, including the aforementioned ones and others, we built a model that assigned the author, as human or AI, at well over 99% accuracy, resulting in 20 times fewer misclassified documents compared to the field-leading approach. This strategy for discriminating a particular set of humans writing from AI could be further adapted and developed by others with basic skills in supervised classification, enabling access to many highly accurate and targeted models for detecting AI usage in academic writing and beyond.
翻译:ChatGPT使大众能够接触AI生成的写作内容,短短数月间,这款产品已颠覆知识经济,引发了人们工作、学习和写作方式的文化变革。区分人类写作与AI写作的需求如今既关键又紧迫,尤其在高等教育和学术写作等领域——此前AI尚未对作者身份构成显著威胁或贡献。为应对这一需求,我们开发了一种基于普遍易获取的监督分类方法,用于区分ChatGPT生成的文本与(人类)学术科学家的写作。我们聚焦于特定人类群体(学术科学家)与ChatGPT写作方式的差异,这种针对性方法发现了区分(这些)人类与AI的新特征;例如,科学家倾向于撰写长段落,且偏好模棱两可的语言,频繁使用"但是""然而""尽管"等词汇。基于包含上述特征在内的20个特征集,我们构建了一个以超过99%准确率判定作者(人类或AI)的模型,其错误分类文档数量比领域领先方法减少了20倍。这种区分特定人类写作群体与AI的策略可供具备基础监督分类技能的研究者进一步适配与开发,从而在学术写作及其他领域广泛应用高精度、高针对性的AI使用检测模型。