Extracting fine-grained experimental findings from literature can provide massive utility for scientific applications. Prior work has focused on developing annotation schemas and datasets for limited aspects of this problem, leading to simpler information extraction datasets which do not capture the real-world complexity and nuance required for this task. Focusing on biomedicine, this work presents CARE (Clinical Aggregation-oriented Result Extraction) -- a new IE dataset for the task of extracting clinical findings. We develop a new annotation schema capturing fine-grained findings as n-ary relations between entities and attributes, which includes phenomena challenging for current IE systems such as discontinuous entity spans, nested relations, and variable arity n-ary relations. Using this schema, we collect extensive annotations for 700 abstracts from two sources: clinical trials and case reports. We also benchmark the performance of various state-of-the-art IE systems on our dataset, including extractive models and generative LLMs in fully supervised and limited data settings. Our results demonstrate the difficulty of our dataset -- even SOTA models such as GPT4 struggle, particularly on relation extraction. We release our annotation schema and CARE to encourage further research on extracting and aggregating scientific findings from literature.

翻译：从文献中提取细粒度的实验结果可为科学应用提供巨大价值。以往工作侧重于开发针对该问题有限方面的标注方案和数据集，导致信息提取数据集过于简单，未能捕捉此项任务所需的现实世界复杂性和细微差别。本研究聚焦生物医学领域，提出CARE（面向临床聚合的结果提取）——一个用于提取临床结果的新型信息提取数据集。我们开发了新的标注方案，将细粒度结果捕获为实体与属性之间的n元关系，其中包含当前信息提取系统难以处理的棘手段现象，如不连续实体跨度、嵌套关系及可变元数的n元关系。基于该方案，我们从临床试验和病例报告两个来源的700篇摘要中收集了大规模标注数据。我们还在此数据集上基准测试了多种先进信息提取系统的性能，包括全监督和有限数据设置下的抽取式模型与生成式大语言模型。结果表明我们的数据集具有挑战性——即便是GPT4等最先进模型也表现困难，尤其在关系提取方面。我们公开发布标注方案和CARE数据集，以鼓励进一步开展从文献中提取和聚合科学发现的研究。