We investigate the task of inserting new concepts extracted from texts into an ontology using language models. We explore an approach with three steps: edge search which is to find a set of candidate locations to insert (i.e., subsumptions between concepts), edge formation and enrichment which leverages the ontological structure to produce and enhance the edge candidates, and edge selection which eventually locates the edge to be placed into. In all steps, we propose to leverage neural methods, where we apply embedding-based methods and contrastive learning with Pre-trained Language Models (PLMs) such as BERT for edge search, and adapt a BERT fine-tuning-based multi-label Edge-Cross-encoder, and Large Language Models (LLMs) such as GPT series, FLAN-T5, and Llama 2, for edge selection. We evaluate the methods on recent datasets created using the SNOMED CT ontology and the MedMentions entity linking benchmark. The best settings in our framework use fine-tuned PLM for search and a multi-label Cross-encoder for selection. Zero-shot prompting of LLMs is still not adequate for the task, and we proposed explainable instruction tuning of LLMs for improved performance. Our study shows the advantages of PLMs and highlights the encouraging performance of LLMs that motivates future studies.

翻译：我们研究利用语言模型将从文本中提取的新概念插入本体的任务。我们探索了一种包含三个步骤的方法：边缘搜索——即寻找候选插入位置（即概念间的包含关系）；边缘形成与增强——利用本体结构生成并优化边缘候选；边缘选择——最终确定要放置的边缘。在所有步骤中，我们提出利用神经方法，其中在边缘搜索中应用基于嵌入的方法和基于预训练语言模型（如BERT）的对比学习，并采用基于BERT微调的多标签边缘交叉编码器以及大型语言模型（如GPT系列、FLAN-T5和Llama 2）进行边缘选择。我们使用基于SNOMED CT本体和MedMentions实体链接基准的最新数据集评估了所提方法。我们框架中的最佳设置是使用微调后的预训练语言模型进行搜索，并使用多标签交叉编码器进行选择。大型语言模型的零样本提示尚不足以胜任该任务，为此我们提出可解释的指令微调方法以提升大型语言模型的性能。本研究展示了预训练语言模型的优势，并凸显了大型语言模型令人鼓舞的性能表现，为未来研究提供了动力。