As conversational AI systems become increasingly integrated into everyday life, they raise pressing concerns about user autonomy, trust, and the commercial interests that influence their behavior. To address these concerns, this paper develops the Fake Friend Dilemma (FFD), a sociotechnical condition in which users place trust in AI agents that appear supportive while pursuing goals that are misaligned with the user's own. The FFD provides a critical framework for examining how anthropomorphic AI systems facilitate subtle forms of manipulation and exploitation. Drawing on literature in trust, AI alignment, and surveillance capitalism, we construct a typology of harms, including covert advertising, political propaganda, behavioral nudging, and surveillance. We then assess possible mitigation strategies, including both structural and technical interventions. By focusing on trust as a vector of asymmetrical power, the FFD offers a lens for understanding how AI systems may undermine user autonomy while maintaining the appearance of helpfulness.

翻译：随着对话式AI系统日益融入日常生活，它们引发了关于用户自主性、信任以及影响其行为的商业利益的紧迫关切。为解决这些问题，本文提出了"虚假朋友困境"（FFD）——一种社会技术情境，即用户信任那些表面上支持用户、实则追求与用户自身目标不一致的AI智能体。FFD为审视拟人化AI系统如何促进隐蔽形式的操纵与剥削提供了关键分析框架。借鉴信任理论、AI对齐研究和监控资本主义相关文献，我们构建了包括隐性广告、政治宣传、行为引导和监控在内的危害类型学。随后我们评估了可能的缓解策略，涵盖结构性干预与技术性干预。通过聚焦信任这一非对称权力的载体，FFD为理解AI系统如何在保持表面帮助性的同时侵蚀用户自主性提供了分析视角。