Context: Test case prioritization (TCP) is a technique widely used by software development organizations to accelerate regression testing. Objectives: We aim to systematize existing TCP knowledge and to propose and empirically evaluate a new TCP approach. Methods: We conduct a snowballing review (SR) on TCP, implement a~comprehensive platform for TCP research (TCPFramework), analyze existing evaluation metrics and propose two new ones (\rAPFDc{} and ATR), and develop a~family of ensemble TCP methods called approach combinators. Results: The SR helped identify 324 studies related to TCP. The techniques proposed in our study were evaluated on the RTPTorrent dataset, consistently outperforming their base approaches across the majority of subject programs, and achieving performance comparable to the current state of the art for heuristical algorithms (in terms of \rAPFDc{}, NTR, and ATR), while using a distinct approach. Conclusions: The proposed methods can be used efficiently for TCP, reducing the time spent on regression testing by up to 2.7\%. Approach combinators offer significant potential for improvements in future TCP research, due to their composability.
翻译:背景:测试用例优先级排序(TCP)是软件开发组织广泛用于加速回归测试的技术。目标:我们旨在系统化现有TCP知识,提出并实证评估一种新的TCP方法。方法:我们对TCP进行了滚雪球式综述(SR),实现了用于TCP研究的综合平台(TCPFramework),分析了现有评估指标并提出了两个新指标(\rAPFDc{}和ATR),开发了称为方法组合子的集成TCP方法族。结果:SR帮助识别了324项与TCP相关的研究。我们研究中提出的技术在RTPTorrent数据集上进行了评估,在大多数被测程序中持续优于其基础方法,并在启发式算法方面(就\rAPFDc{}、NTR和ATR而言)取得了与当前最先进技术相当的性能,同时采用了独特的方法。结论:所提出的方法可有效用于TCP,将回归测试时间减少高达2.7%。方法组合子因其可组合性,为未来TCP研究的改进提供了显著潜力。