Background: Systematic literature reviews (SLRs) have become prevalent in software engineering research. Several researchers may conduct SLRs on similar topics without a prospective register for SLR protocols. However, even ignoring these unavoidable duplications of effort in the simultaneous conduct of SLRs, the proliferation of overlapping and often repetitive SLRs indicates that researchers are not extensively checking for existing SLRs on a topic. Given how effort-intensive it is to design, conduct, and report an SLR, the situation is less than ideal for software engineering research. Aim: To understand how authors justify additional SLRs on a topic. Method: To illustrate the issue and develop suggestions for improvement to address this issue, we have intentionally picked a sufficiently narrow but well-researched topic, i.e., effort estimation in Agile software development. We identify common justification patterns through a qualitative content analysis of 18 published SLRs. We further consider the citation data, publication years, publication venues, and the quality of the SLRs when interpreting the results. Results: The common justification patterns include authors claiming gaps in coverage, methodological limitations in prior studies, temporal obsolescence of previous SLRs, or rapid technological and methodological advancements necessitating updated syntheses. Conclusion: Our in-depth analysis of SLRs on a fairly narrow topic provides insights into SLRs in software engineering in general. By emphasizing the need for identifying existing SLRs and for justifying the undertaking of further SLRs, both in design and review guidelines and as a policy of conferences and journals, we can reduce the likelihood of duplication of effort and increase the rate of progress in the field.
翻译:背景:系统性文献综述在软件工程研究中已日益普遍。多名研究者可能在缺乏前瞻性综述方案注册的情况下,对相似主题开展系统性文献综述。然而,即使忽略这些在同步开展系统性文献综述过程中不可避免的重复劳动,大量重叠且往往重复的系统性文献综述表明,研究者并未充分核查特定主题下已有的系统性文献综述。鉴于设计、实施和报告系统性文献综述所需的高强度工作量,这种现状对软件工程研究领域而言并非理想状态。目的:探究作者如何论证针对同一主题开展额外系统性文献综述的必要性。方法:为阐明该问题并提出改进建议,我们有意选取了一个范围足够狭窄但研究充分的主题——敏捷软件开发中的工作量估算。通过对18篇已发表系统性文献综述的质性内容分析,我们识别出常见的论证模式。在结果解读过程中,我们进一步综合考量了引文数据、出版年份、出版渠道以及系统性文献综述的质量。结果:常见的论证模式包括:作者声称存在覆盖空白、先前研究存在方法论局限、过往系统性文献综述已随时间过时,或快速发展的技术方法需要更新的综合论述。结论:我们对这一相对狭窄主题下系统性文献综述的深度分析,为理解软件工程领域系统性文献综述的整体状况提供了启示。通过在设计与评审指南中,以及作为会议和期刊的政策,强调识别现有系统性文献综述的必要性,并为开展进一步系统性文献综述提供充分论证,我们能够降低重复劳动的可能性,从而提升该领域的研究进展速率。