Peer review shapes which scientific claims enter the published record, but its internal dynamics are hard to measure at scale because reviewer criticism and author revision are usually embedded in long, unstructured correspondence. Here we use a fixed-prompt large language model pipeline to convert the review correspondence of \textit{Nature Communications} papers published from 2017 to 2024 into structured reviewer--author interactions. We find that review pressure is concentrated in the first round and focused disproportionately on core claims rather than peripheral presentation. Higher average opinion strength is also associated with more reviewer disagreement, while review patterns vary little with broad team attributes, consistent with relatively impartial evaluation. Contrary to the intuition that stronger papers should pass review more smoothly, with greater reviewer--author agreement and less extensive revision, we find that stronger criticism, higher-quality comments, and greater revision burden are associated with higher later citation impact within accepted papers. We finally show that fields differ more in review style than in review length, pointing to disciplinary variation in how criticism is negotiated and resolved. These findings position open peer review not just as a gatekeeping mechanism but as a measurable record of how influential scientific claims are challenged, defended, and revised before entering the published record.
翻译:同行评议决定了哪些科学主张能够进入正式出版的学术记录,但其内在机制难以大规模衡量——因为审稿人的批评意见与作者的修改过程通常隐含在冗长且非结构化的通信中。本研究采用固定提示的大语言模型流水线,将2017至2024年间《自然·通讯》期刊论文的评审通信转化为结构化的审稿人-作者交互数据。研究发现:评审压力集中于首轮审稿,且显著聚焦于论文核心主张而非边缘表述;较高的平均观点强度与更大的审稿人分歧相关,而评审模式随团队整体属性的变化甚微,这符合相对公正的评审特征。与“高质量论文应更顺利通过评审”的直觉相反(即审稿人与作者分歧更少、修订幅度更小),我们发现:在已发表的论文中,更尖锐的批评、更高质的意见及更大的修订负担反而与后续更高的引文影响力相关。最后,不同学科在评审风格上的差异显著大于评审篇幅差异,这揭示了各学科在批评协商与解决方式上的差异性。这些发现将开放同行评议定位为不仅是准入机制,更是科学主张在进入正式记录前如何被质疑、辩护及修改的可量化记录。