Motivation: Named Entity Recognition (NER) is a key task to support biomedical research. In Biomedical Named Entity Recognition (BioNER), obtaining high-quality expert annotated data is laborious and expensive, leading to the development of automatic approaches such as distant supervision. However, manually and automatically generated data often suffer from the unlabeled entity problem, whereby many entity annotations are missing, degrading the performance of full annotation NER models. Results: To address this problem, we systematically study the effectiveness of partial annotation learning methods for biomedical entity recognition over different simulated scenarios of missing entity annotations. Furthermore, we propose a TS-PubMedBERT-Partial-CRF partial annotation learning model. We harmonize 15 biomedical NER corpora encompassing five entity types to serve as a gold standard and compare against two commonly used partial annotation learning models, BiLSTM-Partial-CRF and EER-PubMedBERT, and the state-of-the-art full annotation learning BioNER model PubMedBERT tagger. Results show that partial annotation learning-based methods can effectively learn from biomedical corpora with missing entity annotations. Our proposed model outperforms alternatives and, specifically, the PubMedBERT tagger by 38% in F1-score under high missing entity rates. The recall of entity mentions in our model is also competitive with the upper bound on the fully annotated dataset.
翻译:动机:命名实体识别(NER)是支撑生物医学研究的关键任务。在生物医学命名实体识别(BioNER)中,获取高质量专家标注数据既耗时又昂贵,因此推动了远程监督等自动化方法的发展。然而,人工与自动生成的数据常面临未标注实体问题,即大量实体标注缺失,从而降低了全标注NER模型的性能。结果:为解决该问题,我们系统研究了不同模拟实体标注缺失场景下部分标注学习方法在生物医学实体识别中的有效性。此外,我们提出了一种TS-PubMedBERT-Partial-CRF部分标注学习模型。我们整合了涵盖五种实体类型的15个生物医学NER语料库作为金标准,并与两种常用部分标注学习模型(BiLSTM-Partial-CRF和EER-PubMedBERT)以及当前最先进的全标注学习BioNER模型PubMedBERT标签器进行了对比。结果表明,基于部分标注学习的方法能够有效利用存在实体标注缺失的生物医学语料库进行学习。在实体标注高度缺失的情况下,我们提出的模型在所有备选方案中表现最优,特别是其F1值相比PubMedBERT标签器提升了38%。此外,该模型对实体提及的召回率也与全标注数据集的上限具有竞争力。