Given the dominance of dense retrievers that do not generalize well beyond their training dataset distributions, domain-specific test sets are essential in evaluating retrieval. There are few test datasets for retrieval systems intended for use by healthcare providers in a point-of-care setting. To fill this gap we have collaborated with medical professionals to create CURE, an ad-hoc retrieval test dataset for passage ranking with 2000 queries spanning 10 medical domains with a monolingual (English) and two cross-lingual (French/Spanish -> English) conditions. In this paper, we describe how CURE was constructed and provide baseline results to showcase its effectiveness as an evaluation tool. CURE is published with a Creative Commons Attribution Non Commercial 4.0 license and can be accessed on Hugging Face.

翻译：鉴于密集检索器在其训练数据分布之外泛化能力不足的现状，领域特定的测试集对于检索评估至关重要。目前，面向医疗从业者在临床诊疗场景下使用的检索系统，其测试数据集较为匮乏。为填补这一空白，我们与医学专家合作构建了CURE，这是一个用于段落排序的即席检索测试数据集，包含2000个查询，覆盖10个医学领域，并设置了单语（英语）及两种跨语言（法语/西班牙语→英语）检索条件。本文阐述了CURE的构建过程，并提供了基线实验结果以验证其作为评估工具的有效性。CURE采用知识共享署名非商业性4.0许可协议发布，用户可通过Hugging Face平台获取。