Generative AI (GenAI) is increasingly being integrated into the online ecosystem, including online health communities (OHCs), where people with diverse health conditions exchange social support. For example, in OHCs, support providers are beginning to share content generated, directly or indirectly, by popular GenAI-based tools. OHCs are governed by norms that define appropriate behavior when providing support. Ways in which AI-generated support interacts with these norms remain underexplored. Inappropriate conformance or outright violation can erode seekers' trust, distort decision-making, and threaten community sustenance. In this work, we examine whether (and how) AI-generated support conforms to norms, using popular opioid-use recovery subreddits as our testbed. First, we provide an inventory of norms regulating text-based support provision in OHCs. Next, using human-validated LLM judges, we assess the prevalence of AI's conformity to these norms. Finally, through an expert review, we identify risks to seekers (and OHCs) resulting from norm (non)conformity. Our analysis revealed that, while AI-generated support conforms to norms, such conformity may be inappropriate or insufficient, for example, by over- or under-validating seekers in distress. Moreover, we observed instances of outright norm violation. This work provides insights that can help moderators and OHC designers adapt existing and develop new norms to regulate AI integration, protecting both seekers and communities they rely on.
翻译:生成式人工智能(GenAI)正日益融入在线生态系统,包括在线健康社区(OHCs),在这些社区中,患有不同健康状况的人们交流社会支持。例如,在OHCs中,支持提供者开始分享由流行的GenAI工具直接或间接生成的内容。OHCs受规范约束,这些规范定义了提供支持时的适当行为。AI生成的支持与这些规范的互动方式尚待深入探索。不当的遵从或直接违反可能会削弱寻求者的信任、扭曲决策,并威胁社区的持续发展。在本研究中,我们检查了AI生成的支持是否(以及如何)符合规范,使用流行的阿片类药物康复子论坛作为测试平台。首先,我们提供了管理OHCs中基于文本的支持提供的规范清单。接着,使用经过人工验证的LLM评估员,我们评估了AI对这些规范的遵从程度。最后,通过专家评审,我们识别了由于规范(不)遵从对寻求者(和OHCs)带来的风险。我们的分析揭示,虽然AI生成的支持符合规范,但这种符合可能不适当或不充分,例如,过度或不足地验证了处于困境中的寻求者。此外,我们观察到直接违反规范的实例。本研究提供了见解,有助于版主和OHC设计者适应现有规范并制定新规范以管理AI集成,保护寻求者及其所依赖的社区。