Context: Modern software development increasingly relies on software testing for an ever more frequent delivery of high quality software. This puts high demands on the quality of the central artifacts in software testing, test suites and test cases. Objective: We aim to develop a comprehensive model for capturing the dimensions of test case/suite quality, which are relevant for a variety of perspectives. Method: We have carried out a systematic literature review to identify and analyze existing secondary studies on quality aspects of software testing artifacts. Results: We identified 49 relevant secondary studies. Of these 49 studies, less than half did some form of quality appraisal of the included primary studies and only 3 took into account the quality of the primary study when synthesizing the results. We present an aggregation of the context dimensions and factors that can be used to characterize the environment in which the test case/suite quality is investigated. We also provide a comprehensive model of test case/suite quality with definitions for the quality attributes and measurements based on findings in the literature and ISO/IEC 25010:2011. Conclusion: The test artifact quality model presented in the paper can be used to support test artifact quality assessment and improvement initiatives in practice. Furtherm Information and Software Technology 139 (2021): 106620ore, the model can also be used as a framework for documenting context characteristics to make research results more accessible for research and practice.
翻译:背景:现代软件开发愈发依赖软件测试,以支持更频繁地交付高质量软件。这对测试核心工件(即测试套件与测试用例)的质量提出了高要求。目标:我们旨在开发一个综合模型,以捕获测试用例/套件质量的多维度特性,使其适用于多种视角。方法:我们开展了一项系统性文献综述,识别并分析了现有关于软件测试工件质量方面的二级研究。结果:我们找到了49篇相关的二级研究。在这49项研究中,不到一半对纳入的原始研究进行了某种形式的质量评估,仅有3项在综合结果时考虑了原始研究的质量。我们整合了可用于表征测试用例/套件质量研究环境的情境维度与因素。同时,基于文献发现及ISO/IEC 25010:2011标准,我们提出了一个包含质量属性定义与测量指标的全面测试用例/套件质量模型。结论:本文提出的测试工件质量模型可用于支持实践中测试工件的质量评估与改进活动。此外,该模型还可作为记录情境特征的框架,使研究成果更易于被研究与实践领域所获取和应用。