Large language models (LLMs) can reproduce a wide variety of rhetorical styles and generate text that expresses a broad spectrum of sentiments. This capacity, now available at low cost, makes them powerful tools for manipulation and control. In this paper, we consider a set of underestimated societal harms made possible by the rapid and largely unregulated adoption of LLMs. Rather than consider LLMs as isolated digital artefacts used to displace this or that area of work, we focus on the large-scale computational infrastructure upon which they are instrumentalised across domains. We begin with discussion on how LLMs may be used to both pollute and uniformize information environments and how these modalities may be leveraged as mechanisms of control. We then draw attention to several areas of emerging research, each of which compounds the capabilities of LLMs as instruments of power. These include (i) persuasion through the real-time design of choice architectures in conversational interfaces (e.g., via "AI personas"), (ii) the use of LLM-agents as computational models of human agents (e.g., "silicon subjects"), (iii) the use of LLM-agents as computational models of human agent populations (e.g., "silicon societies") and finally, (iv) the combination of LLMs with reinforcement learning to produce controllable and steerable strategic dialogue models. We draw these strands together to discuss how these areas may be combined to build LLM-based systems that serve as powerful instruments of individual, social and political control via the simulation and disingenuous "prediction" of human behaviour, intent, and action.

翻译：大型语言模型（LLMs）能够复现多种修辞风格，并生成表达广泛情感倾向的文本。这种如今以低成本可获取的能力，使其成为操纵与控制的强大工具。本文探讨了因LLMs快速且基本不受监管的应用而可能引发的一系列被低估的社会危害。我们并未将LLMs视为取代特定工作领域的孤立数字制品，而是聚焦于支撑其跨领域工具化应用的大规模计算基础设施。首先讨论LLMs如何被用于污染与同质化信息环境，以及这些模式如何被用作控制机制。随后我们关注数个新兴研究领域，每个领域都增强了LLMs作为权力工具的能力，包括：（i）通过对话界面中实时设计的抉择架构进行说服（如借助"人工智能人格"），（ii）将LLM智能体用作人类代理的计算模型（如"硅基主体"），（iii）将LLM智能体用作人类群体代理的计算模型（如"硅基社会"），以及（iv）将LLMs与强化学习相结合，生成可控且可引导的策略性对话模型。我们将这些线索整合起来，探讨如何融合这些领域构建基于LLM的系统，通过模拟与伪称性的"预测"人类行为、意图与行动，使其成为个体、社会及政治控制的强大工具。