Automated scholarly paper review (ASPR) has entered the coexistence phase with traditional peer review, where artificial intelligence (AI) systems are increasingly incorporated into real-world manuscript evaluation. In parallel, research on automated and AI-assisted peer review has proliferated. Despite this momentum, empirical progress remains constrained by several critical limitations in existing datasets. While reviewers routinely evaluate figures, tables, and complex layouts to assess scientific claims, most existing datasets remain overwhelmingly text-centric. This bias is reinforced by a narrow focus on data from computer science venues. Furthermore, these datasets lack precise alignment between reviewer comments and specific manuscript versions, obscuring the iterative relationship between peer review and manuscript evolution. In response, we introduce FMMD, a multimodal and multidisciplinary open peer review dataset curated from F1000Research. The dataset bridges the current gap by integrating manuscript-level visual and structural data with version-specific reviewer reports and editorial decisions. By providing explicit alignment between reviewer comments and the exact article iteration under review, FMMD enables fine-grained analysis of the peer review lifecycle across diverse scientific domains. FMMD supports tasks such as multimodal issue detection and multimodal review comment generation. It provides a comprehensive empirical resource for the development of peer review research.
翻译:自动化论文评审(ASPR)已进入与传统同行评审共存的阶段,人工智能(AI)系统正日益融入现实稿件评审流程。与此同时,自动化和AI辅助同行评审的研究也迅速增长。尽管发展势头迅猛,但实证研究仍受限于现有数据集的多重关键缺陷。虽然评审人通常通过评估图表、表格和复杂版式来验证科学主张,但现有数据集绝大多数仍以文本为中心。这种偏差因过度依赖计算机科学领域的数据而进一步加剧。此外,这些数据集缺乏评审意见与特定稿件版本间的精确对应关系,掩盖了同行评审与稿件演进之间的迭代关联。为此,我们推出FMMD——一个基于F1000Research构建的多模态跨学科开放同行评审数据集。该数据集通过整合稿件层级的视觉与结构数据、版本特定的评审报告及编辑决策,弥补了当前空白。通过建立评审意见与被审文章具体迭代版本的显式关联,FMMD支持跨学科领域的同行评审生命周期细粒度分析。本数据集支持多模态问题检测、多模态评审意见生成等任务,为同行评审研究的发展提供了全面的实证资源。