Mentions of new concepts appear regularly in texts and require automated approaches to harvest and place them into Knowledge Bases (KB), e.g., ontologies and taxonomies. Existing datasets suffer from three issues, (i) mostly assuming that a new concept is pre-discovered and cannot support out-of-KB mention discovery; (ii) only using the concept label as the input along with the KB and thus lacking the contexts of a concept label; and (iii) mostly focusing on concept placement w.r.t a taxonomy of atomic concepts, instead of complex concepts, i.e., with logical operators. To address these issues, we propose a new benchmark, adapting MedMentions dataset (PubMed abstracts) with SNOMED CT versions in 2014 and 2017 under the Diseases sub-category and the broader categories of Clinical finding, Procedure, and Pharmaceutical / biologic product. We provide usage on the evaluation with the dataset for out-of-KB mention discovery and concept placement, adapting recent Large Language Model based methods.

翻译：新概念提及在文本中频繁出现，需要自动化方法将其提取并置于知识库（KB，如本体和分类体系）中。现有数据集存在三个问题：(i) 大多假设新概念已被预先发现，无法支持知识库外的提及发现；(ii) 仅将概念标签与知识库作为输入，缺乏概念标签的上下文信息；(iii) 主要关注原子概念在分类体系中的放置，而非复杂概念（即包含逻辑运算符的概念）。为解决这些问题，我们提出一个新的基准数据集，基于MedMentions数据集（PubMed摘要），适配了SNOMED CT在2014年和2017年版本中疾病子类别以及临床发现、操作流程和药物/生物制品等更广泛类别。我们提供了将该数据集用于知识库外提及发现与概念放置的评估方法，并适配了基于最新大语言模型的方法。