Software testing is a mandatory activity in any serious software development process, as bugs are a reality in software development. This raises the question of quality: good tests are effective in finding bugs, but until a test case actually finds a bug, its effectiveness remains unknown. Therefore, determining what constitutes a good or bad test is necessary. This is not a simple task, and there are a number of studies that identify different characteristics of a good test case. A previous study evaluated 29 hypotheses regarding what constitutes a good test case, but the findings are based on developers' beliefs, which are subjective and biased. In this paper we investigate eight of these hypotheses, through an extensive empirical study based on open software repositories. Despite our best efforts, we were unable to find evidence that supports these beliefs. This indicates that, although these hypotheses represent good software engineering advice, they do not necessarily mean that they are enough to provide the desired outcome of good testing code.
翻译:软件测试是任何严肃软件开发过程中不可或缺的活动,因为缺陷是软件开发中的现实问题。这引出了质量的问题:优秀的测试能有效发现缺陷,但在测试用例实际发现缺陷之前,其有效性仍是未知的。因此,确定何为好测试或坏测试是必要的。这不是一项简单的任务,已有若干研究识别了优秀测试用例的不同特征。先前一项研究评估了关于何为优秀测试用例的29项假设,但研究结论基于开发者的信念,而信念具有主观性和偏差性。本文通过基于开源软件仓库的广泛实证研究,对其中八项假设进行了探究。尽管我们尽了最大努力,仍未能找到支持这些信念的证据。这表明,虽然这些假设代表了良好的软件工程建议,但它们未必足以产生预期的高质量测试代码。