Large Language Models (LLMs) can generate highly persuasive text, raising concerns about their misuse for propaganda, manipulation, and other harmful purposes. This leads us to our central question: Is LLM-generated persuasion more difficult to automatically detect than human-written persuasion? To address this, we categorize controllable generation approaches for producing persuasive content with LLMs and introduce Persuaficial, a high-quality multilingual benchmark covering six languages: English, German, Polish, Italian, French and Russian. Using this benchmark, we conduct extensive empirical evaluations comparing human-authored and LLM-generated persuasive texts. We find that although overtly persuasive LLM-generated texts can be easier to detect than human-written ones, subtle LLM-generated persuasion consistently degrades automatic detection performance. Beyond detection performance, we provide the first comprehensive linguistic analysis contrasting human and LLM-generated persuasive texts, offering insights that may guide the development of more interpretable and robust detection tools.

翻译：大型语言模型（LLMs）能够生成极具说服力的文本，这引发了对其可能被滥用于宣传、操纵及其他有害目的的担忧。这引出了我们的核心问题：与人类撰写的劝说文本相比，LLM生成的劝说文本是否更难被自动检测？为此，我们对利用LLMs生成劝说内容的可控生成方法进行了分类，并提出了Persuaficial——一个高质量的多语言基准测试集，涵盖英语、德语、波兰语、意大利语、法语和俄语六种语言。利用该基准，我们进行了广泛的实证评估，比较了人类撰写和LLM生成的劝说文本。研究发现，虽然显性的LLM生成劝说文本可能比人类撰写的更容易检测，但隐性的LLM生成劝说文本会持续降低自动检测的性能。除了检测性能，我们首次提供了对比人类与LLM生成劝说文本的全面语言学分析，这些发现可能为开发更具可解释性和鲁棒性的检测工具提供指导。