Existing fact-checking models for biomedical claims are typically trained on synthetic or well-worded data and hardly transfer to social media content. This mismatch can be mitigated by adapting the social media input to mimic the focused nature of common training claims. To do so, Wuehrl & Klinger (2022) propose to extract concise claims based on medical entities in the text. However, their study has two limitations: First, it relies on gold-annotated entities. Therefore, its feasibility for a real-world application cannot be assessed since this requires detecting relevant entities automatically. Second, they represent claim entities with the original tokens. This constitutes a terminology mismatch which potentially limits the fact-checking performance. To understand both challenges, we propose a claim extraction pipeline for medical tweets that incorporates named entity recognition and terminology normalization via entity linking. We show that automatic NER does lead to a performance drop in comparison to using gold annotations but the fact-checking performance still improves considerably over inputting the unchanged tweets. Normalizing entities to their canonical forms does, however, not improve the performance.
翻译:现有针对生物医学声明的事实核查模型通常基于合成数据或措辞严谨的数据训练,难以迁移至社交媒体内容。这种不匹配可通过调整社交媒体输入以模拟常见训练声明的聚焦特性来缓解。为此,Wuehrl & Klinger(2022)提出基于文本中的医学实体提取简洁声明。然而其研究存在两处局限:其一,该方法依赖人工标注实体,故无法评估其在真实世界应用中的可行性——因为此类应用需自动检测相关实体;其二,他们使用原始词元表征声明实体,构成术语不匹配问题,可能限制事实核查性能。为理解这两项挑战,我们提出面向医疗推文的声明提取流水线,该流水线整合了命名实体识别与基于实体链接的术语规范化。实验表明:相比使用人工标注,自动NER确实导致性能下降,但相较于直接输入未修改推文,事实核查性能仍显著提升;将实体规范化至规范形式则未带来性能改善。