Biomedical claim verification fails if no evidence can be discovered. In these cases, the fact-checking verdict remains unknown and the claim is unverifiable. To improve upon this, we have to understand if there are any claim properties that impact its verifiability. In this work we assume that entities and relations define the core variables in a biomedical claim's anatomy and analyze if their properties help us to differentiate verifiable from unverifiable claims. In a study with trained annotation experts we prompt them to find evidence for biomedical claims, and observe how they refine search queries for their evidence search. This leads to the first corpus for scientific fact verification annotated with subject-relation-object triplets, evidence documents, and fact-checking verdicts (the BEAR-Fact corpus). We find (1) that discovering evidence for negated claims (e.g., X-does-not-cause-Y) is particularly challenging. Further, we see that annotators process queries mostly by adding constraints to the search and by normalizing entities to canonical names. (2) We compare our in-house annotations with a small crowdsourcing setting where we employ medical experts and laypeople. We find that domain expertise does not have a substantial effect on the reliability of annotations. Finally, (3), we demonstrate that it is possible to reliably estimate the success of evidence retrieval purely from the claim text~(.82\F), whereas identifying unverifiable claims proves more challenging (.27\F). The dataset is available at http://www.ims.uni-stuttgart.de/data/bioclaim.

翻译：生物医学声明验证在无法发现证据时会失败。在这些情况下，事实核查结论未知，声明被视为不可验证。为改进这一状况，我们需要理解是否存在影响声明可验证性的属性。本文假设实体和关系是生物医学声明解剖结构中的核心变量，并分析其属性是否有助于区分可验证与不可验证的声明。在一项由训练有素的标注专家参与的研究中，我们引导他们寻找生物医学声明的证据，并观察他们如何优化搜索查询以寻找证据。由此产生了首个标注了主语-关系-宾语三元组、证据文档和事实核查结论的科学事实验证语料库（BEAR-Fact语料库）。我们发现：（1）发现否定声明（例如“X不会导致Y”）的证据尤为困难。此外，观察到标注者主要通过向搜索添加约束条件以及将实体标准化为规范名称来处理查询。（2）我们将内部标注与一个涉及医学专家和非专业人士的小规模众包设置进行了比较。发现领域专业知识对标注的可靠性并无显著影响。最后，（3）我们证明仅从声明文本即可可靠估计证据检索的成功率（.82\F），而识别不可验证的声明则更具挑战性（.27\F）。数据集可在 http://www.ims.uni-stuttgart.de/data/bioclaim 获取。