The technology of Conversational AI has made significant advancements over the last eighteen months. As a consequence, conversational agents are likely to be deployed in the near future that are designed to pursue targeted influence objectives. Sometimes referred to as the "AI Manipulation Problem," the emerging risk is that consumers will unwittingly engage in real-time dialog with predatory AI agents that can skillfully persuade them to buy particular products, believe particular pieces of misinformation, or fool them into revealing sensitive personal data. For many users, current systems like ChatGPT and LaMDA feel safe because they are primarily text-based, but the industry is already shifting towards real-time voice and photorealistic digital personas that look, move, and express like real people. This will enable the deployment of agenda-driven Virtual Spokespeople (VSPs) that will be highly persuasive through real-time adaptive influence. This paper explores the manipulative tactics that are likely to be deployed through conversational AI agents, the unique threats such agents pose to the epistemic agency of human users, and the emerging need for policymakers to protect against the most likely predatory practices.

翻译：在过去十八个月中，对话式人工智能技术取得了显著进展。因此，近期可能部署旨在追求特定影响目标的对话代理。这种被称为“人工智能操纵问题”的新兴风险在于，消费者可能会在不知情的情况下与具有掠夺性的AI代理进行实时对话，这些代理能够巧妙地说服他们购买特定产品、相信特定虚假信息，或诱骗其泄露敏感个人数据。对许多用户而言，ChatGPT和LaMDA等现有系统因主要基于文本交互而显得安全，但该行业已开始转向实时语音和逼真的数字人形象——这些形象在外观、动作和表情上如同真人一般。这将使得以议程为导向的虚拟代言人（Virtual Spokespeople，VSPs）得以部署，通过实时自适应影响实现高度说服力。本文探讨了可能通过对话式AI代理部署的操纵策略、此类代理对人类用户认知能动性构成的独特威胁，以及政策制定者保护用户免受最可能发生的掠夺性行为侵害的新兴需求。