Peer review, the bedrock of scientific advancement in machine learning (ML), is strained by a crisis of scale. Exponential growth in manuscript submissions to premier ML venues such as NeurIPS, ICML, and ICLR is outpacing the finite capacity of qualified reviewers, leading to concerns about review quality, consistency, and reviewer fatigue. This position paper argues that AI-assisted peer review must become an urgent research and infrastructure priority. We advocate for a comprehensive AI-augmented ecosystem, leveraging Large Language Models (LLMs) not as replacements for human judgment, but as sophisticated collaborators for authors, reviewers, and Area Chairs (ACs). We propose specific roles for AI in enhancing factual verification, guiding reviewer performance, assisting authors in quality improvement, and supporting ACs in decision-making. Crucially, we contend that the development of such systems hinges on access to more granular, structured, and ethically-sourced peer review process data. We outline a research agenda, including illustrative experiments, to develop and validate these AI assistants, and discuss significant technical and ethical challenges. We call upon the ML community to proactively build this AI-assisted future, ensuring the continued integrity and scalability of scientific validation, while maintaining high standards of peer review.
翻译:同行评审作为机器学习(ML)领域科学进步的基石,正面临规模危机的严峻挑战。在NeurIPS、ICML和ICLR等顶级ML会议中,投稿数量的指数级增长已超越合格审稿人的有限承载能力,引发关于评审质量、一致性和审稿人倦怠的担忧。本文立场声明认为,AI辅助同行评审必须成为亟待研究与基础设施建设的优先事项。我们倡导构建全面的AI增强型生态系统,在利用大语言模型(LLMs)时,并非将其作为人类判断的替代品,而是作为作者、审稿人和领域主席(ACs)的智能协作工具。我们提出AI在增强事实核查、引导审稿人表现、协助作者提升稿件质量以及支持AC决策等环节的具体职责。至关重要的是,我们认为此类系统的开发依赖于获取更细粒度、结构化且符合伦理规范的同行评审过程数据。我们制定了包含示例性实验的研究路线图以开发验证这些AI助手,并深入探讨了重大技术与伦理挑战。我们呼吁ML社区主动构建这一AI辅助的未来,在维护高标准的同行评审体系前提下,确保科学验证的持续完整性与可扩展性。