ChatGPT has gained significant interest due to its impressive performance, but people are increasingly concerned about its potential risks, particularly around the detection of AI-generated content (AIGC), which is often difficult for untrained humans to identify. Current datasets utilized for detecting ChatGPT-generated text primarily center around question-answering, yet they tend to disregard tasks that possess semantic-invariant properties, such as summarization, translation, and paraphrasing. Our primary studies demonstrate that detecting model-generated text on semantic-invariant tasks is more difficult. To fill this gap, we introduce a more extensive and comprehensive dataset that considers more types of tasks than previous work, including semantic-invariant tasks. In addition, the model after a large number of task instruction fine-tuning shows a strong powerful performance. Owing to its previous success, we further instruct fine-tuning T\textit{k}-instruct and build a more powerful detection system.
翻译:ChatGPT因其卓越性能而备受关注,但人们日益担忧其潜在风险,尤其是难以被未经训练的普通人识别的人工智能生成内容(AIGC)检测问题。当前用于检测ChatGPT生成文本的数据集主要集中在问答领域,但往往忽略了摘要、翻译和释义等具有语义不变特性的任务。我们的初步研究表明,检测模型在语义不变任务上生成的文本更为困难。为填补这一空白,我们引入了比以往工作更广泛、更全面的数据集,涵盖了包括语义不变任务在内的多种任务类型。此外,经过大量任务指令微调的模型展现出强大的性能。鉴于其先前取得的成功,我们进一步对T\textit{k}-instruct进行指令微调,构建了更强大的检测系统。