Systematic reviews in medicine play a critical role in evidence-based decision-making by aggregating findings from multiple studies. A central bottleneck in automating this process is extracting numeric evidence and determining study-level conclusions for specific outcomes and comparisons. Prior work has framed this problem as a textual inference task by retrieving relevant content fragments and inferring conclusions from them. However, such approaches often rely on shallow textual cues and fail to capture the underlying numeric reasoning behind expert assessments. In this work, we conceptualise the problem as one of quantitative reasoning. Rather than inferring conclusions from surface text, we extract structured numerical evidence (e.g., event counts or standard deviations) and apply domain knowledge informed logic to derive outcome-specific conclusions. We develop a numeric reasoning system composed of a numeric data extraction model and an effect estimate component, enabling more accurate and interpretable inference aligned with the domain expert principles. We train the numeric data extraction model using different strategies, including supervised fine-tuning (SFT) and reinforcement learning (RL) with a new value reward model. When evaluated on the CochraneForest benchmark, our best-performing approach -- using RL to train a small-scale number extraction model -- yields up to a 21% absolute improvement in F1 score over retrieval-based systems and outperforms general-purpose LLMs of over 400B parameters by up to 9% on the RCTs benchmark. Our results demonstrate the promise of reasoning-driven approaches for automating systematic evidence synthesis.

翻译：医学领域的系统综述通过整合多项研究结果，在循证决策中发挥着关键作用。自动化该过程的核心瓶颈在于提取数值证据，并为特定结局指标和比较项确定研究级结论。先前研究通过检索相关内容片段并从中推断结论，将该问题构建为文本推理任务。然而，此类方法通常依赖浅层文本线索，未能捕捉专家评估背后的深层数值推理机制。本研究将该问题概念化为定量推理任务：我们并非从表层文本推断结论，而是提取结构化数值证据（如事件计数或标准差），并应用基于领域知识的逻辑规则来推导特定结局的结论。我们开发了一个由数值数据提取模型和效应估计模块组成的数值推理系统，能够依据领域专家原则实现更准确、可解释的推断。我们采用多种策略训练数值数据提取模型，包括监督微调（SFT）和基于新型价值奖励模型的强化学习（RL）。在CochraneForest基准测试中，我们性能最佳的方法——使用RL训练的小规模数值提取模型——相比基于检索的系统在F1分数上实现了高达21%的绝对提升，并在RCTs基准测试中以高达9%的优势超越参数量超过400B的通用大语言模型。我们的研究结果证明了推理驱动方法在自动化系统证据合成方面的巨大潜力。