Context: The constant growth of primary evidence and Systematic Literature Reviews (SLRs) publications in the Software Engineering (SE) field leads to the need for SLR Updates. However, searching and selecting evidence for SLR updates demands significant effort from SE researchers. Objective: We present emerging results on an automated approach to support searching and selecting studies for SLR updates in SE. Method: We developed an automated tool prototype to perform the snowballing search technique and support selecting relevant studies for SLR updates using Machine Learning (ML) algorithms. We evaluated our automation proposition through a small-scale evaluation with a reliable dataset from an SLR replication and its update. Results: Effectively automating snowballing-based search strategies showed feasibility with minor losses, specifically related to papers without Digital Object Identifier (DOI). The ML algorithm giving the highest performance to select studies for SLR updates was Linear Support Vector Machine, with approximately 74% recall and 15% precision. Using such algorithms with conservative thresholds to minimize the risk of missing papers can significantly reduce evidence selection efforts. Conclusion: The preliminary results of our evaluation point in promising directions, indicating the potential of automating snowballing search efforts and of reducing the number of papers to be manually analyzed by about 2.5 times when selecting evidence for updating SLRs in SE.
翻译:背景:软件工程领域中原始证据及系统文献综述(SLR)出版物的持续增长,导致了对SLR更新的需求。然而,为SLR更新搜索和选取证据需要软件工程研究者投入大量精力。目的:我们展示了一种自动化方法在软件工程SLR更新中支持研究搜索与选取的新进展。方法:我们开发了一个自动化工具原型,通过机器学习(ML)算法实现滚雪球搜索技术,并支持SLR更新中相关研究的选取。我们利用来自SLR重复研究及其更新的可靠数据集进行小规模评估,验证了自动化方案的可行性。结果:有效自动化基于滚雪球的搜索策略显示出可行性,仅在与缺失数字对象标识符(DOI)的论文相关时出现微小损失。在选取SLR更新研究时性能最高的ML算法是线性支持向量机,其召回率约为74%,精确率约为15。采用保守阈值以降低遗漏论文风险的此类算法,可显著减少证据选取的工作量。结论:本次评估的初步结果指向了有前景的方向,表明自动化滚雪球搜索的潜力,以及在为软件工程SLR更新选取证据时,可将需人工分析的论文数量减少约2.5倍。