The emergence of large language models (LLMs), such as Generative Pre-trained Transformer 4 (GPT-4) used by ChatGPT, has profoundly impacted the academic and broader community. While these models offer numerous advantages in terms of revolutionizing work and study methods, they have also garnered significant attention due to their potential negative consequences. One example is generating academic reports or papers with little to no human contribution. Consequently, researchers have focused on developing detectors to address the misuse of LLMs. However, most existing methods prioritize achieving higher accuracy on restricted datasets, neglecting the crucial aspect of generalizability. This limitation hinders their practical application in real-life scenarios where reliability is paramount. In this paper, we present a comprehensive analysis of the impact of prompts on the text generated by LLMs and highlight the potential lack of robustness in one of the current state-of-the-art GPT detectors. To mitigate these issues concerning the misuse of LLMs in academic writing, we propose a reference-based Siamese detector named Synthetic-Siamese which takes a pair of texts, one as the inquiry and the other as the reference. Our method effectively addresses the lack of robustness of previous detectors (OpenAI detector and DetectGPT) and significantly improves the baseline performances in realistic academic writing scenarios by approximately 67% to 95%.

翻译：大语言模型（LLMs）的出现，例如ChatGPT所使用的生成式预训练变换器4（GPT-4），已对学术界及更广泛的社群产生了深远影响。尽管这些模型在革新工作与学习方法方面具有诸多优势，但它们也因其潜在负面后果而备受关注。例如，生成几乎或完全没有人类贡献的学术报告或论文。因此，研究者们致力于开发检测器以应对LLMs的滥用问题。然而，现有方法大多优先追求在受限数据集上取得更高准确率，忽视了泛化能力这一关键方面。这一局限性阻碍了它们在可靠性至上的实际场景中的应用。本文全面分析了提示对LLMs生成文本的影响，并揭示了当前最先进的GPT检测器之一可能存在的鲁棒性不足。为缓解学术写作中LLMs滥用的相关问题，我们提出了一种基于参考的孪生检测器，名为Synthetic-Siamese，该检测器接受一对文本，分别作为查询文本和参考文本。我们的方法有效解决了先前检测器（OpenAI检测器与DetectGPT）鲁棒性不足的问题，并在真实学术写作场景中将基线性能显著提升约67%至95%。