Automated disinformation generation is often listed as one of the risks of large language models (LLMs). The theoretical ability to flood the information space with disinformation content might have dramatic consequences for democratic societies around the world. This paper presents a comprehensive study of the disinformation capabilities of the current generation of LLMs to generate false news articles in English language. In our study, we evaluated the capabilities of 10 LLMs using 20 disinformation narratives. We evaluated several aspects of the LLMs: how well they are at generating news articles, how strongly they tend to agree or disagree with the disinformation narratives, how often they generate safety warnings, etc. We also evaluated the abilities of detection models to detect these articles as LLM-generated. We conclude that LLMs are able to generate convincing news articles that agree with dangerous disinformation narratives.

翻译：自动化虚假信息生成常被列为大型语言模型（LLMs）的风险之一。在信息空间充斥虚假内容的潜在能力可能对全球民主社会产生严重后果。本文对当前一代LLMs生成英文虚假新闻文章的虚假信息能力进行了全面研究。在我们的研究中，我们使用20种虚假信息叙事评估了10个LLMs的能力。我们评估了LLMs的多个方面：它们生成新闻文章的能力、与虚假信息叙事保持一致或抵触的倾向强度、生成安全警告的频率等。我们还评估了检测模型识别这些文章为LLM生成内容的能力。我们得出结论：LLMs能够生成与危险虚假信息叙事相一致的令人信服的新闻文章。