The rapid adoption of generative language models has brought about substantial advancements in digital communication, while simultaneously raising concerns regarding the potential misuse of AI-generated content. Although numerous detection methods have been proposed to differentiate between AI and human-generated content, the fairness and robustness of these detectors remain underexplored. In this study, we evaluate the performance of several widely-used GPT detectors using writing samples from native and non-native English writers. Our findings reveal that these detectors consistently misclassify non-native English writing samples as AI-generated, whereas native writing samples are accurately identified. Furthermore, we demonstrate that simple prompting strategies can not only mitigate this bias but also effectively bypass GPT detectors, suggesting that GPT detectors may unintentionally penalize writers with constrained linguistic expressions. Our results call for a broader conversation about the ethical implications of deploying ChatGPT content detectors and caution against their use in evaluative or educational settings, particularly when they may inadvertently penalize or exclude non-native English speakers from the global discourse.
翻译:生成式语言模型的快速应用在数字通信领域带来了显著进步,同时也引发了关于AI生成内容潜在滥用的担忧。尽管已有多种检测方法被提出用于区分AI生成内容与人类创作内容,但这些检测器的公平性和鲁棒性尚未得到充分探究。本研究通过对比母语与非母语英语写作者的写作样本,评估了多种广泛使用的GPT检测器的性能。研究结果揭示,这些检测器一致地将非母语英语写作样本误判为AI生成内容,而母语写作样本则能被准确识别。此外,研究表明,简单的提示策略不仅能够缓解这种偏见,还能有效绕过GPT检测器,暗示GPT检测器可能无意中对语言表达受限的写作者造成惩罚。我们的研究结果呼吁对部署ChatGPT内容检测器的伦理影响展开更广泛的讨论,并警示在评估或教育环境中使用此类检测器的风险,特别是在它们可能无意中惩罚或排斥非母语英语使用者参与全球讨论的背景下。