Peer review constitutes a core component of scholarly publishing; yet it demands substantial expertise and training, and is susceptible to errors and biases. Various applications of NLP for peer reviewing assistance aim to support reviewers in this complex process, but the lack of clearly licensed datasets and multi-domain corpora prevent the systematic study of NLP for peer review. To remedy this, we introduce NLPeer -- the first ethically sourced multidomain corpus of more than 5k papers and 11k review reports from five different venues. In addition to the new datasets of paper drafts, camera-ready versions and peer reviews from the NLP community, we establish a unified data representation and augment previous peer review datasets to include parsed and structured paper representations, rich metadata and versioning information. We complement our resource with implementations and analysis of three reviewing assistance tasks, including a novel guided skimming task. Our work paves the path towards systematic, multi-faceted, evidence-based study of peer review in NLP and beyond. The data and code are publicly available.

翻译：同行评议是学术出版的核心环节，但这一过程需要丰富的专业知识和训练，且容易出现错误和偏见。自然语言处理（NLP）在同行评议辅助领域的多项应用旨在支持评审者完成这一复杂流程，然而，缺乏明确许可的数据集和多领域语料库阻碍了NLP在同行评议中的系统性研究。为此，我们提出了NLPeer——首个来自五个不同会议场所、包含超过5000篇论文和11000份评审报告、并遵循伦理准则获取的多领域语料库。除新增来自NLP社区的论文初稿、终稿及同行评议数据集外，我们还建立了统一的数据表示标准，并通过补充解析后的结构化论文表征、丰富的元数据和版本信息来增强现有同行评议数据集。我们通过实现和分析三项评审辅助任务（包括一项创新的引导式速览任务）来完善这一资源。本研究为NLP领域及更广范围内开展系统性、多维度、基于证据的同行评议研究铺平了道路。相关数据与代码已公开提供。