The content moderation systems used by social media sites are a topic of widespread interest and research, but less is known about the use of similar systems by web search engines. For example, Google Search attempts to help its users navigate three distinct types of data voids--when the available search results are deemed low-quality, low-relevance, or rapidly-changing--by placing one of three corresponding warning banners at the top of the search page. Here we collected 1.4M unique search queries shared on social media to surface Google's warning banners, examine when and why those banners were applied, and train deep learning models to identify data voids beyond Google's classifications. Across three data collection waves (Oct 2023, Mar 2024, Sept 2024), we found that Google returned a warning banner for about 1% of our search queries, with substantial churn in the set of queries that received a banner across waves. The low-quality banners, which warn users that their results "may not have reliable information on this topic," were especially rare, and their presence was associated with low-quality domains in the search results and conspiracy-related keywords in the search query. Low-quality banner presence was also inconsistent over short time spans, even when returning highly similar search results. In August 2024, low-quality banners stopped appearing on the SERPs we collected, but average search result quality remained largely unchanged, suggesting they may have been discontinued by Google. Using our deep learning models to analyze both queries and search results in context, we identify 29 to 58 times more low-quality data voids than there were low-quality banners, and find a similar number after the banners had disappeared. Our findings point to the need for greater transparency on search engines' content moderation practices, especially around important events like elections.

翻译：社交媒体平台采用的内容审核系统已成为广泛关注与研究的话题，然而网络搜索引擎对类似系统的运用却鲜为人知。例如，Google搜索试图通过在其搜索结果页顶部展示三种对应的警示横幅之一，来帮助用户应对三类不同的数据空白——即当现有搜索结果被判定为质量低下、相关性不足或快速变化时。本研究收集了社交媒体上分享的140万条独立搜索查询，以揭示Google警示横幅的出现规律，探究其应用时机与原因，并训练深度学习模型以识别超出Google分类范围的数据空白。通过三轮数据采集（2023年10月、2024年3月、2024年9月），我们发现Google对约1%的搜索查询返回了警示横幅，且各轮次中获得横幅的查询集合存在显著变动。其中，提示用户“此主题可能没有可靠信息”的低质量横幅尤为罕见，其出现与搜索结果中的低质量域名及搜索查询中的阴谋论关键词密切相关。即使在返回高度相似搜索结果的情况下，低质量横幅在短时间内的出现也缺乏一致性。2024年8月，低质量横幅在我们采集的搜索结果页中停止出现，但平均搜索结果质量基本保持不变，暗示Google可能已终止该功能。通过运用深度学习模型对查询语句及上下文搜索结果进行分析，我们识别出的低质量数据空白数量是低质量横幅的29至58倍，且在横幅消失后仍检测到类似规模的数据空白。我们的研究结果表明，搜索引擎需要提高内容审核实践的透明度，尤其在选举等重要事件期间。