Large language model (LLM) based chatbots show promise in persuasive communication, but existing studies often rely on weak controls or focus on belief change rather than behavioral intentions or outcomes. This pre-registered multi-country (US, Canada, UK) randomized controlled trial involving 930 vaccine-hesitant parents evaluated brief (three-minute) multi-turn conversations with LLM-based chatbots against standard public health messaging approaches for increasing human papillomavirus (HPV) vaccine intentions for their children. Participants were randomly assigned to: (1) a weak control (no message), (2) a strong control reflecting the standard of care (reading official public health materials), or (3 and 4) one of two chatbot conditions. One chatbot was prompted to deliver short, conversational responses, while the other used the model's default output style (longer with bullet points). While chatbot interactions significantly increased self-reported vaccination intent (by 7.1-10.3 points on a 100-point scale) compared to no message, they did not outperform standard public health materials, with the conversational chatbot performing significantly worse. Additionally, while the short-term effects of chatbot interactions faded during a 15-day follow-up, the effects of public health material persisted through a 45-day follow-up relative to no message. These findings suggest that while LLMs can effectively shift vaccination intentions in the short-term, their incremental value over existing public health communications is questionable, offering a more tempered view of their persuasive capabilities and highlighting the importance of integrating AI-driven tools alongside, rather than replacing, current public health strategies.

翻译：基于大语言模型（LLM）的聊天机器人在说服性沟通中展现出潜力，但现有研究常依赖弱对照组或聚焦于信念改变而非行为意向或实际结果。这项预注册的多国（美国、加拿大、英国）随机对照试验纳入了930名对疫苗接种持犹豫态度的家长，评估了基于LLM的聊天机器人进行简短（三分钟）多轮对话与标准公共卫生信息传递方法在提升其子女人乳头瘤病毒（HPV）疫苗接种意愿方面的效果。参与者被随机分配至：（1）弱对照组（无信息干预），（2）强对照组（反映现行标准处理方式——阅读官方公共卫生材料），或（3与4）两种聊天机器人干预组之一。其中一个聊天机器人被设定为输出简短对话式回应，另一个则采用模型默认输出风格（较长且带项目符号）。结果显示，与无信息干预相比，聊天机器人互动显著提升了自我报告的疫苗接种意愿（在100分量表上提高7.1-10.3分），但并未优于标准公共卫生材料，且对话式聊天机器人的表现显著更差。此外，聊天机器人干预的短期效应在15天随访期内逐渐消退，而公共卫生材料相对于无信息干预的效果在45天随访期内持续存在。这些发现表明，尽管LLM能在短期内有效改变疫苗接种意愿，但其相对于现有公共卫生沟通方式的增量价值尚存疑问，这为LLM的说服能力提供了更为审慎的评估视角，并凸显了将人工智能驱动工具作为现有公共卫生策略的补充而非替代的重要性。