The rapid advancement of AI companionship systems has positioned them as scalable interventions for addressing social isolation. Current design approaches emphasize maximizing user engagement and satisfaction, treating effective alignment between AI capabilities and user needs as an unqualified success. However, this framing may overlook a critical dimension of bidirectional human-AI alignment: when AI systems successfully align with users' expressed emotional needs, users may reciprocally adapt their relational expectations in ways that undermine authentic human connection and agency. We examine what we term the authenticity paradox: the phenomenon whereby successful bidirectional alignment in emotionally supportive AI paradoxically harms the values that motivated the intervention. Through the analysis of AI companionship for older adults as an illustrative case, we identify four key tensions that emerge when technical effectiveness generates ethical concerns: the dilemma of AI becoming users' only accessible option, mismatches between emotional needs and system-level interventions, conflicts over sense of control during vulnerable moments, and fundamental disagreements about whose values should guide system behavior.
翻译:人工智能伴侣系统的快速发展使其成为解决社会隔离的可扩展干预措施。当前设计方法强调最大化用户参与度和满意度,将AI能力与用户需求的有效对齐视为毋庸置疑的成功。然而,这种框架可能忽略了人机双向对齐的关键维度:当AI系统成功对齐用户表达的情感需求时,用户可能以损害真实人类连接和能力的方式反向调整其关系期望。我们审视了所谓的真实性悖论现象:情感支持型AI中成功的双向对齐反而损害了干预初衷所珍视的价值。以老年群体AI伴侣分析为例,我们识别出技术有效性引发伦理问题的四种关键张力:AI成为用户唯一可用选项的困境、情感需求与系统层面干预的不匹配、脆弱时刻控制感的冲突,以及关于何种价值观应主导系统行为的根本分歧。