Google Search increasingly surfaces AI-generated content through features like AI Overviews (AIO) and Featured Snippets (FS), which users frequently rely on despite having no control over their presentation. Through a systematic algorithm audit of 1,508 real baby care and pregnancy-related queries, we evaluate the quality and consistency of these information displays. Our robust evaluation framework assesses multiple quality dimensions, including answer consistency, relevance, presence of medical safeguards, source categories, and sentiment alignment. Our results reveal concerning gaps in information consistency, with information in AIO and FS displayed on the same search result page being inconsistent with each other in 33% of cases. Despite high relevance scores, both features critically lack medical safeguards (present in just 11% of AIO and 7% of FS responses). While health and wellness websites dominate source categories for both, AIO and FS, FS also often link to commercial sources. These findings have important implications for public health information access and demonstrate the need for stronger quality controls in AI-mediated health information. Our methodology provides a transferable framework for auditing AI systems across high-stakes domains where information quality directly impacts user well-being.
翻译:随着AI概览(AIO)和精选片段(FS)等功能的推出,谷歌搜索结果中越来越多地呈现AI生成内容,用户对此高度依赖却无法控制其呈现方式。通过对1508个真实婴儿护理与妊娠相关查询的系统性算法审计,我们评估了这些信息展示的质量与一致性。我们的稳健评估框架涵盖多个质量维度,包括答案一致性、相关性、医疗安全措施存在性、来源类别及情感匹配度。结果显示信息一致性存在显著缺口:在同一搜索结果页面中,33%的案例中AIO与FS展示的信息互不一致。尽管相关性评分较高,但两者均严重缺乏医疗安全措施(仅11%的AIO和7%的FS回复包含此类措施)。虽然健康养生类网站在AIO和FS的来源类别中均占主导地位,但FS还频繁链接至商业来源。这些发现对公共卫生信息获取具有重要启示,表明需加强对AI介导健康信息的质量控制。我们的方法论为在高风险领域(信息质量直接影响用户福祉)审计AI系统提供了可迁移框架。