The advent of LLMs has given rise to a new type of web search: Generative search, where LLMs retrieve web pages related to a query and generate a single, coherent text as a response. This output modality stands in stark contrast to traditional web search, where results are returned as a ranked list of independent web pages. In this paper, we ask: Along what dimensions do generative search outputs differ from traditional web search? We compare Google, a traditional web search engine, with four generative search engines from two providers (Google and OpenAI) across queries from four domains. Our analysis reveals intriguing differences. Most generative search engines cover a wider range of sources compared to web search. Generative search engines vary in the degree to which they rely on internal knowledge contained within the model parameters v.s. external knowledge retrieved from the web. Generative search engines surface varying sets of concepts, creating new opportunities for enhancing search diversity and serendipity. Our results also highlight the need for revisiting evaluation criteria for web search in the age of Generative AI.

翻译：大型语言模型（LLM）的出现催生了一种新型网络搜索模式：生成式搜索。该模式下，LLM会检索与查询相关的网页并生成连贯的文本作为响应。这种输出方式与传统网络搜索形成鲜明对比——传统搜索以独立网页的排序列表形式返回结果。本文旨在探究：生成式搜索输出在哪些维度上与传统网络搜索存在差异？我们选取传统搜索引擎Google与两家提供商（Google和OpenAI）的四款生成式搜索引擎，通过四个领域的查询进行对比分析。研究发现若干显著差异：多数生成式搜索引擎比传统网络搜索覆盖更广泛的信源；不同生成式搜索引擎在依赖模型参数内部知识与检索网络外部知识方面存在程度差异；生成式搜索引擎呈现的概念集合具有多样性，为提升搜索多样性与意外发现创造了新机遇。本研究结果同时表明，在生成式人工智能时代，有必要重新审视网络搜索的评估标准。