Systematic reviews and meta-analyses rely on converting narrative articles into structured, numerically grounded study records. Despite rapid advances in large language models (LLMs), it remains unclear whether they can meet the structural requirements of this process, which hinge on preserving roles, methods, and effect-size attribution across documents rather than on recognizing isolated entities. We propose a structural, diagnostic framework that evaluates LLM-based evidence extraction as a progression of schema-constrained queries with increasing relational and numerical complexity, enabling precise identification of failure points beyond atom-level extraction. Using a manually curated corpus spanning five scientific domains, together with a unified query suite and evaluation protocol, we evaluate two state-of-the-art LLMs under both per-document and long-context, multi-document input regimes. Across domains and models, performance remains moderate for single-property queries but degrades sharply once tasks require stable binding between variables, roles, statistical methods, and effect sizes. Full meta-analytic association tuples are extracted with near-zero reliability, and long-context inputs further exacerbate these failures. Downstream aggregation amplifies even minor upstream errors, rendering corpus-level statistics unreliable. Our analysis shows that these limitations stem not from entity recognition errors, but from systematic structural breakdowns, including role reversals, cross-analysis binding drift, instance compression in dense result sections, and numeric misattribution, indicating that current LLMs lack the structural fidelity, relational binding, and numerical grounding required for automated meta-analysis. The code and data are publicly available at GitHub (https://github.com/zhiyintan/LLM-Meta-Analysis).

翻译：系统综述与荟萃分析依赖于将叙述性文章转换为结构化、基于数值的研究记录。尽管大语言模型（LLMs）取得了快速进展，但其是否能满足这一过程的结构性要求仍不明确——该过程的关键在于跨文档保持角色、方法和效应量归因的稳定性，而非识别孤立实体。我们提出一个结构化的诊断框架，将基于LLM的证据提取评估为一系列模式约束查询的递进过程，这些查询具有递增的关系和数值复杂度，从而能够精确识别原子级提取之外的故障点。利用一个涵盖五个科学领域的人工标注语料库，结合统一的查询套件和评估协议，我们在单文档输入和长上下文多文档输入两种模式下评估了两个最先进的LLM。跨领域和模型的分析表明，单属性查询的性能保持中等水平，但一旦任务需要在变量、角色、统计方法和效应量之间建立稳定绑定，性能便急剧下降。完整的荟萃分析关联元组的提取可靠性近乎为零，而长上下文输入进一步加剧了这些故障。下游聚合甚至放大了轻微的上游错误，导致语料库层面的统计结果不可靠。我们的分析表明，这些局限并非源于实体识别错误，而是源于系统性的结构崩溃，包括角色反转、跨分析绑定漂移、密集结果部分的实例压缩以及数值归因错误，这表明当前LLM缺乏自动化荟萃分析所需的结构保真度、关系绑定能力和数值基础。代码与数据已在GitHub（https://github.com/zhiyintan/LLM-Meta-Analysis）公开。