Formulating selective information needs results in queries that implicitly specify set operations, such as intersection, union, and difference. For instance, one might search for "shorebirds that are not sandpipers" or "science-fiction films shot in England". To study the ability of retrieval systems to meet such information needs, we construct QUEST, a dataset of 3357 natural language queries with implicit set operations, that map to a set of entities corresponding to Wikipedia documents. The dataset challenges models to match multiple constraints mentioned in queries with corresponding evidence in documents and correctly perform various set operations. The dataset is constructed semi-automatically using Wikipedia category names. Queries are automatically composed from individual categories, then paraphrased and further validated for naturalness and fluency by crowdworkers. Crowdworkers also assess the relevance of entities based on their documents and highlight attribution of query constraints to spans of document text. We analyze several modern retrieval systems, finding that they often struggle on such queries. Queries involving negation and conjunction are particularly challenging and systems are further challenged with combinations of these operations.

翻译：选择性信息需求的表述会导致查询中隐式指定集合操作，例如交集、并集和差集。例如，用户可能搜索"不是矶鹬的滨鸟"或"在英国拍摄的科幻电影"。为研究检索系统满足此类信息需求的能力，我们构建了QUEST数据集，包含3357条含隐式集合操作的自然语言查询，这些查询映射到对应维基百科文档的实体集合。该数据集要求模型将查询中提及的多个约束条件与文档中的相应证据进行匹配，并正确执行多种集合操作。数据集采用半自动化方式构建，利用维基百科类别名称自动组合各独立类别生成查询，再由众包工作者进行释义优化并验证其自然度与流畅性。众包工作者还根据文档评估实体的相关性，并标注查询约束条件与文档文本片段的对应关系。我们分析了多种现代检索系统，发现这些系统在此类查询上往往表现不佳。涉及否定与合取运算的查询尤为困难，当组合使用这些操作时，系统面临更大挑战。