We wish to measure the information coverage of an ad hoc retrieval algorithm, that is, how much of the range of available relevant information is covered by the search results. Information coverage is a central aspect for retrieval, especially when the retrieval system is integrated with generative models in a retrieval-augmented generation (RAG) system. The classic metrics for ad hoc retrieval, precision and recall, reward a system as more and more relevant documents are retrieved. However, since relevance in ad hoc test collections is defined for a document without any relation to other documents that might contain the same information, high recall is sufficient but not necessary to ensure coverage. The same is true for other metrics such as rank-biased precision (RBP), normalized discounted cumulative gain (nDCG), and mean average precision (MAP). Test collections developed around the notion of diversity ranking in web search incorporate multiple aspects that support a concept of coverage in the web domain. In this work, we construct a suite of collections for evaluating information coverage from existing collections. This suite offers researchers a unified testbed spanning multiple genres and tasks. All topics, nuggets, relevance labels, and baseline rankings are released on Hugging Face Datasets, along with instructions for accessing the publicly available document collections.

翻译：我们旨在衡量即兴检索算法的信息覆盖范围，即搜索结果涵盖可用相关信息的广度。信息覆盖是检索的核心维度，尤其当检索系统与生成模型集成于检索增强生成（RAG）系统时更为关键。即兴检索的经典指标——精确率与召回率——会随着更多相关文档被检索而奖励系统。然而，由于即兴测试集合中相关性定义仅针对单个文档，与可能包含相同信息的其他文档无关联，高召回率虽足以确保覆盖，却非必要条件。其他指标如排名偏置精确率（RBP）、归一化折损累计增益（nDCG）及平均精度均值（MAP）亦存在类似局限。围绕Web搜索中多样性排序概念构建的测试集合，通过纳入多个维度支持了Web领域的覆盖概念。本研究基于现有集合构建了一套评估信息覆盖的测试集体系。该体系为研究者提供了横跨多种体裁与任务的统一测试平台。所有主题、信息片段、相关性标签及基线排序均已发布至Hugging Face数据集，并附有公开文档集合的访问指南。