Thanks to the state-of-the-art Large Language Models (LLMs), language generation has reached outstanding levels. These models are capable of generating high quality content, thus making it a challenging task to detect generated text from human-written content. Despite the advantages provided by Natural Language Generation, the inability to distinguish automatically generated text can raise ethical concerns in terms of authenticity. Consequently, it is important to design and develop methodologies to detect artificial content. In our work, we present some classification models constructed by ensembling transformer models such as Sci-BERT, DeBERTa and XLNet, with Convolutional Neural Networks (CNNs). Our experiments demonstrate that the considered ensemble architectures surpass the performance of the individual transformer models for classification. Furthermore, the proposed SciBERT-CNN ensemble model produced an F1-score of 98.36% on the ALTA shared task 2023 data.

翻译：得益于最先进的大语言模型（LLMs），文本生成已达到卓越水平。这些模型能够生成高质量内容，使得区分机器生成文本与人工撰写内容成为一项具有挑战性的任务。尽管自然语言生成为我们带来诸多便利，但无法自动识别生成文本可能会引发真实性方面的伦理问题。因此，设计并开发检测人工内容的方法具有重要意义。本研究提出了若干分类模型，这些模型通过集成Sci-BERT、DeBERTa和XLNet等Transformer模型与卷积神经网络（CNNs）构建。实验表明，所提出的集成架构在分类性能上超越了单一Transformer模型。此外，所提出的SciBERT-CNN集成模型在2023年ALTA共享任务数据集上取得了98.36%的F1分数。