Large language models (LLMs) have shown remarkable performance in various tasks and have been extensively utilized by the public. However, the increasing concerns regarding the misuse of LLMs, such as plagiarism and spamming, have led to the development of multiple detectors, including fine-tuned classifiers and statistical methods. In this study, we equip LLMs with prompts, rather than relying on an external paraphraser, to evaluate the vulnerability of these detectors. We propose a novel Substitution-based In-Context example Optimization method (SICO) to automatically construct prompts for evading the detectors. SICO is cost-efficient as it requires only 40 human-written examples and a limited number of LLM inferences to generate a prompt. Moreover, once a task-specific prompt has been constructed, it can be universally used against a wide range of detectors. Extensive experiments across three real-world tasks demonstrate that SICO significantly outperforms the paraphraser baselines and enables GPT-3.5 to successfully evade six detectors, decreasing their AUC by 0.5 on average. Furthermore, a comprehensive human evaluation as well as a validation experiment in the wild show that the SICO-generated text achieves human-level readability and task completion rates. Finally, the strong performance of SICO exhibits its potential as a reliable evaluation tool for future detectors. The codes and data are located on https://github.com/ColinLu50/Evade-GPT-Detector.

翻译：大型语言模型（LLMs）在各类任务中展现出卓越性能，已被公众广泛使用。然而，针对LLMs滥用（如剽窃和垃圾信息）的担忧日益增加，促使研究者开发了包括微调分类器和统计方法在内的多种检测工具。本研究直接为LLMs配备提示词（而非依赖外部改写器），以评估这些检测工具的脆弱性。我们提出一种创新的基于替换的上下文示例优化方法（SICO），可自动构建提示词以规避检测。SICO具有成本效益，仅需40个人工编写的示例及有限次LLM推理即可生成提示词。更关键的是，一旦构建出任务特定提示词，即可泛化应用于多种检测工具。在三个真实任务中的广泛实验表明，SICO显著优于基于改写器的基线方法，使GPT-3.5成功规避六种检测工具，平均降低其AUC值0.5。此外，综合人工评估与野外验证实验显示，SICO生成文本达到人类水平的可读性与任务完成率。最后，SICO的强劲性能展现出其作为未来检测工具可靠性评估工具的潜力。代码与数据已在https://github.com/ColinLu50/Evade-GPT-Detector公开。