Persuasion is a key aspect of what it means to be human, and is central to business, politics, and other endeavors. Advancements in artificial intelligence (AI) have produced AI systems that are capable of persuading humans to buy products, watch videos, click on search results, and more. Even systems that are not explicitly designed to persuade may do so in practice. In the future, increasingly anthropomorphic AI systems may form ongoing relationships with users, increasing their persuasive power. This paper investigates the uncertain future of persuasive AI systems. We examine ways that AI could qualitatively alter our relationship to and views regarding persuasion by shifting the balance of persuasive power, allowing personalized persuasion to be deployed at scale, powering misinformation campaigns, and changing the way humans can shape their own discourse. We consider ways AI-driven persuasion could differ from human-driven persuasion. We warn that ubiquitous highlypersuasive AI systems could alter our information environment so significantly so as to contribute to a loss of human control of our own future. In response, we examine several potential responses to AI-driven persuasion: prohibition, identification of AI agents, truthful AI, and legal remedies. We conclude that none of these solutions will be airtight, and that individuals and governments will need to take active steps to guard against the most pernicious effects of persuasive AI.

翻译：说服是人类本质的关键方面，在商业、政治及其他领域占据核心地位。人工智能的进步催生了能够说服人类购买产品、观看视频、点击搜索结果等行为的AI系统。即使未明确设计具有说服功能的系统，在实践中也可能产生此类效果。未来，日益拟人化的AI系统可能持续与用户建立关系，从而增强其说服能力。本文探究具有说服力的AI系统的不确定性前景。我们分析了AI如何通过改变说服力量平衡、实现大规模个性化说服、助推虚假信息宣传、重塑人类塑造自身话语的方式，从而在本质上改变人类对说服的认知与关系。我们比较了AI驱动型说服与人类驱动型说服的差异，警告称普遍存在的高说服力AI系统可能根本性地改变信息环境，导致人类丧失对自身未来的掌控。为此，我们研究了应对AI驱动型说服的潜在方案：禁止性措施、AI智能体识别、可信AI构建及法律救济途径。结论指出，上述方案均非绝对有效，个人与政府需采取主动措施抵御有害说服型AI的最恶劣影响。