Recent LLMs are able to generate high-quality multilingual texts, indistinguishable for humans from authentic human-written ones. Research in machine-generated text detection is however mostly focused on the English language and longer texts, such as news articles, scientific papers or student essays. Social-media texts are usually much shorter and often feature informal language, grammatical errors, or distinct linguistic items (e.g., emoticons, hashtags). There is a gap in studying the ability of existing methods in detection of such texts, reflected also in the lack of existing multilingual benchmark datasets. To fill this gap we propose the first multilingual (22 languages) and multi-platform (5 social media platforms) dataset for benchmarking machine-generated text detection in the social-media domain, called MultiSocial. It contains 472,097 texts, of which about 58k are human-written and approximately the same amount is generated by each of 7 multilingual LLMs. We use this benchmark to compare existing detection methods in zero-shot as well as fine-tuned form. Our results indicate that the fine-tuned detectors have no problem to be trained on social-media texts and that the platform selection for training matters.

翻译：近期的大型语言模型能够生成高质量的多语言文本，其与真实人类撰写文本的区分度对人类而言已微乎其微。然而，机器生成文本检测的研究目前主要集中于英语及较长文本（如新闻文章、科学论文或学生作文）。社交媒体文本通常篇幅短小，且常包含非正式语言、语法错误或独特的语言要素（例如表情符号、话题标签）。现有方法对此类文本的检测能力研究存在空白，这也反映在缺乏多语言基准数据集上。为填补这一空白，我们提出了首个面向社交媒体领域的多语言（22种语言）、多平台（5个社交媒体平台）机器生成文本检测基准数据集，命名为MultiSocial。该数据集包含472,097条文本，其中约5.8万条为人类撰写，另有大致等量的文本分别由7个多语言大型语言模型生成。我们利用此基准比较了现有检测方法在零样本及微调模式下的性能。结果表明，经过微调的检测器能够顺利适应社交媒体文本的训练，且训练平台的选择对检测效果具有重要影响。