Peer review is a widely accepted mechanism for research evaluation, playing a pivotal role in academic publishing. However, criticisms have long been leveled at this mechanism, mostly because of its poor efficiency and low reproducibility. Recent years have seen the application of artificial intelligence (AI) in assisting the peer review process. Nonetheless, with the involvement of humans, such limitations remain inevitable. In this paper, we propose the concept and pipeline of automated scholarly paper review (ASPR) and review the relevant literature and technologies of achieving a full-scale computerized review process. On the basis of the review and discussion, we conclude that there is already corresponding research and preliminary implementation at each stage of ASPR. We further look into the challenges in ASPR with the existing technologies. The major difficulties lie in inadequate data, imperfect document parsing and representation, defective human$\unicode{x2013}$computer interaction, and flawed deep logical reasoning. Moreover, we point out the future directions and discuss the possible moral and ethical issues of ASPR. In the foreseeable future, ASPR and peer review will coexist in a reinforcing manner before ASPR is able to fully undertake the reviewing workload from humans.
翻译:同行评审是广泛接受的研究评估机制,在学术出版中发挥着关键作用。然而,该机制长期面临批评,主要源于其效率低下和可重复性不足。近年来,人工智能(AI)被应用于辅助同行评审流程。尽管如此,由于人类的参与,这些局限性仍然不可避免。本文提出了自动化学术论文评审(ASPR)的概念与流程,并对实现全流程计算机化评审的相关文献与技术进行了综述。基于综述与讨论,我们得出结论:ASPR的每个阶段均已存在相应的研究与初步实现。我们进一步探讨了现有技术下ASPR面临的挑战,主要困难在于数据不足、文档解析与表征不完善、人机交互存在缺陷以及深层逻辑推理能力欠缺。此外,我们指出了未来发展方向,并讨论了ASPR可能引发的道德伦理问题。在可预见的未来,ASPR将与同行评审以互补共进的方式共存,直至其能够完全承担人类的评审工作量。