For simulation-based systems, finding a set of test cases with the least cost by exploring multiple goals is a complex task. Domain-specific optimization goals (e.g. maximize output variance) are useful for guiding the rapid selection of test cases via mutation. But evaluating the selected test cases via mutation (that can distinguish the current program from something else) is a different goal to domain-specific optimizations. While the optimization goals can be used to guide the mutation analysis, that guidance should be viewed as a weak indicator since it can hurt the mutation effectiveness goals by focusing too much on the optimization goals. Based on the above, this paper proposes DoLesS (Domination with Least Squares Approximation) that selects the minimal and effective test cases by averaging over a coarse-grained grid of the information gained from multiple optimizations goals. DoLesS applies an inverted least squares approximation approach to find a minimal set of tests that can distinguish better from worse parts of the optimization goals. When tested on multiple simulation-based systems, DoLesS performs as well or even better as the prior state-of-the-art, while running 80-360 times faster on average (seconds instead of hours).
翻译:对于基于模拟的系统,通过探索多个目标来寻找成本最低的测试用例集是一项复杂的任务。特定领域的优化目标(例如最大化输出方差)有助于通过变异引导测试用例的快速选择。但通过变异评估所选测试用例(即区分当前程序与其他程序的能力)是与领域特定优化不同的目标。尽管优化目标可用于指导变异分析,但这种指导应被视为弱指标,因为过度关注优化目标可能会损害变异有效性的目标。基于上述问题,本文提出了DoLesS(最小二乘逼近主导方法),该方法通过对多个优化目标在粗粒度网格上获得的信息进行平均,来选择最小且有效的测试用例。DoLesS采用了一种反转最小二乘逼近方法,以找到能够区分优化目标中较好部分与较差部分的最小测试集。在多个基于模拟的系统上进行测试时,DoLesS的表现与先前最先进的方法相当甚至更优,同时平均运行速度快80-360倍(从数小时缩短至数秒)。