AI-powered tools increasingly promise to fill information gaps in health, especially in domains like maternal and reproductive health that demand timely, accurate, and actionable information. This is extremely important, as the United States leads peer nations in preventable deaths, with stark racial disparities. However, current AI and NLP-powered systems aim to improve access to vetted maternal health information by routing user queries to a factual response while under-specifying the socio-technical governance structures that shape trust, use, and harm in practice. We report findings from four synchronous focus groups ($n=24$) with three stakeholder groups central to peripartum information support: birthing people, clinicians, and health workers (e.g., doulas, social workers, community health workers) exploring topics around information seeking, experience with current clinical infrastructure, misinformation, and an AI-enabled factual answering tool design probe. Our inductive analysis surfaces a central finding: in high-stakes health contexts shaped by historical inequities, trustworthiness must be inspectable and not asserted. While stakeholders diverge on what makes information credible, they converge on the need for transparency, recourse, and ecosystem complementarity. Based on the discussions, we identify four themes and governance requirements: (1) support for social and identity-based sensemaking, (2) pluralistic verification practices, (3) inspectable governance with recourse mechanisms, and (4) ecosystem-aware integration that avoids shifting burden. Building on these findings, we propose design artifacts that are mistrust-aware and promote principled governance mechanisms for transparent, pluralistic AI systems. Finally, we discuss the implications of our findings for expanding human-AI evaluations and improving the transparency of deployed AI systems.
翻译:人工智能驱动的工具日益承诺填补健康领域的信息空白,特别是在需要及时、准确和可操作信息的孕产妇及生殖健康等领域,这极其重要。因为美国在可预防死亡方面领先于同类国家,且存在显著的种族差异。然而,当前的AI与自然语言处理(NLP)系统旨在通过将用户查询引导至事实性回复来改善经过审查的孕产妇健康信息的获取,却未充分阐明塑造实践中信任、使用与伤害的社会技术治理结构。我们报告了来自三个围产期信息支持核心利益相关方群体(生育者、临床医生及健康工作者,如导乐、社会工作者、社区健康工作者)的四个同步焦点小组($n=24$)的研究发现,探讨了信息寻求、当前临床基础设施体验、错误信息以及AI赋能的事实性回答工具设计原型等主题。我们的归纳分析浮现出一个核心发现:在由历史不平等塑造的高风险健康情境中,可信赖性必须可审查而非被断言。尽管利益相关方对何为可信信息存在分歧,但他们一致认可透明度、补救机制和生态系统互补性的必要性。基于讨论,我们确定了四个主题及治理需求:(1)支持基于社会与身份的意义构建,(2)多元化的验证实践,(3)具备补救机制的可审查治理,以及(4)避免负担转移的生态系统感知型整合。基于这些发现,我们提出了对不信任敏感的设计制品,并倡导用于透明、多元AI系统的原则化治理机制。最后,我们讨论了研究结果对扩展人-AI评估及提升已部署AI系统透明度的启示。