Mentions of new concepts appear regularly in texts and require automated approaches to harvest and place them into Knowledge Bases (KB), e.g., ontologies and taxonomies. Existing datasets suffer from three issues, (i) mostly assuming that a new concept is pre-discovered and cannot support out-of-KB mention discovery; (ii) only using the concept label as the input along with the KB and thus lacking the contexts of a concept label; and (iii) mostly focusing on concept placement w.r.t a taxonomy of atomic concepts, instead of complex concepts, i.e., with logical operators. To address these issues, we propose a new benchmark, adapting MedMentions dataset (PubMed abstracts) with SNOMED CT versions in 2014 and 2017 under the Diseases sub-category and the broader categories of Clinical finding, Procedure, and Pharmaceutical / biologic product. We provide usage on the evaluation with the dataset for out-of-KB mention discovery and concept placement, adapting recent Large Language Model based methods.

翻译：新概念提及频繁出现在文本中，需要自动化方法将其获取并纳入知识库（如本体和分类体系）。现有数据集存在三个问题：（i）通常假设新概念已被预先发现，无法支持知识库外提及的发现；（ii）仅使用概念标签和知识库作为输入，缺乏概念标签的上下文；（iii）大多聚焦于原子概念分类体系中的概念放置，而非涉及逻辑运算符的复杂概念。为解决这些问题，我们提出新基准测试，基于MedMentions数据集（PubMed摘要）并适配SNOMED CT 2014和2017版本，涵盖疾病子类及临床发现、手术、药品/生物制品等更广泛类别。我们提供了利用该数据集进行知识库外提及发现和概念放置的评估方案，并适配了基于大型语言模型的最新方法。