Large language models can now generate political messages as persuasive as those written by humans, raising concerns about how far this persuasiveness may continue to increase with model size. Here, we generate 720 persuasive messages on 10 U.S. political issues from 24 language models spanning several orders of magnitude in size. We then deploy these messages in a large-scale randomized survey experiment (N = 25,982) to estimate the persuasive capability of each model. Our findings are twofold. First, we find evidence of a log scaling law: model persuasiveness is characterized by sharply diminishing returns, such that current frontier models are barely more persuasive than models smaller in size by an order of magnitude or more. Second, mere task completion (coherence, staying on topic) appears to account for larger models' persuasive advantage. These findings suggest that further scaling model size will not much increase the persuasiveness of static LLM-generated messages.

翻译：大语言模型现已能够生成与人类撰写者同等说服力的政治信息，这引发了关于模型规模持续增大会在多大程度上进一步提升说服力的担忧。本文针对10个美国政治议题，从规模跨越数个数量级的24个语言模型中生成了720条说服性信息。随后，我们通过一项大规模随机化调查实验（样本量=25,982）投放这些信息，以评估每个模型的说服能力。我们的发现包含两个方面：首先，我们发现了对数尺度定律的证据——模型说服力呈现出急剧的收益递减特性，当前前沿模型的说服力仅比规模小一个数量级或更多的模型略胜一筹。其次，较大模型的优势似乎主要源于基本任务完成能力（连贯性、主题一致性）。这些结果表明，进一步扩大模型规模对静态LLM生成信息的说服力提升作用将非常有限。