The rise of LLMs (Large Language Models) has contributed to the improved performance and development of cutting-edge NLP applications. However, these can also pose risks when used maliciously, such as spreading fake news, harmful content, impersonating individuals, or facilitating school plagiarism, among others. This is because LLMs can generate high-quality texts, which are challenging to differentiate from those written by humans. GLTR, which stands for Giant Language Model Test Room and was developed jointly by the MIT-IBM Watson AI Lab and HarvardNLP, is a visual tool designed to help detect machine-generated texts based on GPT-2, that highlights the words in text depending on the probability that they were machine-generated. One limitation of GLTR is that the results it returns can sometimes be ambiguous and lead to confusion. This study aims to explore various ways to improve GLTR's effectiveness for detecting AI-generated texts within the context of the IberLef-AuTexTification 2023 shared task, in both English and Spanish languages. Experiment results show that our GLTR-based GPT-2 model overcomes the state-of-the-art models on the English dataset with a macro F1-score of 80.19%, except for the first ranking model (80.91%). However, for the Spanish dataset, we obtained a macro F1-score of 66.20%, which differs by 4.57% compared to the top-performing model.

翻译：大型语言模型（LLM）的兴起推动了前沿自然语言处理（NLP）应用性能的提升与发展。然而，当这些模型被恶意使用时，也可能带来风险，例如传播虚假新闻、有害内容、冒充个人身份或助长学术抄袭等。这是因为LLM能够生成高质量文本，这些文本难以与人类撰写的文本区分开来。GLTR（全称为巨型语言模型测试室）由MIT-IBM Watson AI实验室与哈佛NLP联合开发，是一种基于GPT-2的视觉化工具，旨在通过高亮显示文本中可能由机器生成概率较高的词汇来辅助检测机器生成文本。GLTR的一个局限性在于其返回的结果有时存在模糊性，可能导致混淆。本研究旨在探索在IberLef-AuTexTification 2023共享任务背景下，提升GLTR检测英语和西班牙语AI生成文本有效性的多种方法。实验结果表明，我们基于GLTR的GPT-2模型在英语数据集上以80.19%的宏观F1分数超越了除排名第一模型（80.91%）外的所有现有最优模型；而在西班牙语数据集上，我们获得了66.20%的宏观F1分数，与最优模型存在4.57%的差距。