Formulating selective information needs results in queries that implicitly specify set operations, such as intersection, union, and difference. For instance, one might search for "shorebirds that are not sandpipers" or "science-fiction films shot in England". To study the ability of retrieval systems to meet such information needs, we construct QUEST, a dataset of 3357 natural language queries with implicit set operations, that map to a set of entities corresponding to Wikipedia documents. The dataset challenges models to match multiple constraints mentioned in queries with corresponding evidence in documents and correctly perform various set operations. The dataset is constructed semi-automatically using Wikipedia category names. Queries are automatically composed from individual categories, then paraphrased and further validated for naturalness and fluency by crowdworkers. Crowdworkers also assess the relevance of entities based on their documents and highlight attribution of query constraints to spans of document text. We analyze several modern retrieval systems, finding that they often struggle on such queries. Queries involving negation and conjunction are particularly challenging and systems are further challenged with combinations of these operations.

翻译：选择性信息需求的表述会生成隐式指定集合操作（如交集、并集和差集）的查询。例如，用户可能搜索"不是矶鹬的滨鸟"或"在英国拍摄的科幻电影"。为研究检索系统满足此类信息需求的能力，我们构建了QUEST数据集——包含3357条带隐式集合操作的自然语言查询，每条查询对应维基百科文档中的一组实体。该数据集要求模型匹配查询中提及的多重约束条件与文档中的对应证据，并正确执行各类集合操作。数据集采用维基百科类别名称进行半自动构建：查询由单个类别自动组合而成，经人工众包改写及自然性、流畅性验证，同时由众包工作者根据文档评估实体相关性，并标注查询约束条件在文档文本中的归属。通过对多个现代检索系统的分析，我们发现这些系统在此类查询上表现欠佳。涉及否定与合取的查询尤为困难，而多种操作的组合进一步加剧了系统的处理难度。