The scientific peer review system has been slowly deteriorating over the last years, and not just within empirical software engineering (ESE) research. Increased submission numbers, high workload, and the rise of generative AI use with all its associated issues have made many cracks in the system more visible. To get a better understanding of the current state of peer review in the ESE community, we conducted a questionnaire survey, which accumulated 120 responses. We report on (i) the perceived review load of community members, (ii) review quality perception as well as frequent challenges for and issues with reviews, (iii) the use of LLM-based tools in the reviewing process, and (iv) the community's suggestions for improving the peer review system. We hope that these community opinions can facilitate more evidence-based discussions about how people want to see the review system change for the better.
翻译:近年来,科学同行评审体系一直在缓慢退化,这不仅仅发生在实证软件工程(ESE)研究领域。投稿数量的增加、繁重的工作负担,以及生成式人工智能应用及其相关问题的涌现,使得该体系的诸多缺陷愈发显现。为了深入了解实证软件工程社区内同行评审的当前状况,我们开展了一项问卷调查,共收集到120份回复。本文报告了:(i)社区成员感知到的评审负荷;(ii)评审质量感知以及评审中经常遇到的挑战与问题;(iii)基于大语言模型的工具在评审过程中的使用情况;(iv)社区对改进同行评审体系的建议。我们希望这些社区意见能推动更多基于证据的讨论,探讨人们希望如何更好地推动评审体系的变革。