Mutation testing is an established software quality assurance technique for the assessment of test suites. While it is well-suited to estimate the general fault-revealing capability of a test suite, it is not practical and informative when the software under test must be validated against specific requirements. This is often the case for embedded software, where the software is typically validated against rigorously-specified safety properties. In such a scenario (i) a mutant is relevant only if it can impact the satisfaction of the tested properties, and (ii) a mutant is meaningfully-killed with respect to a property only if it causes the violation of that property. To address these limitations of mutation testing, we introduce property-based mutation testing, a method for assessing the capability of a test suite to exercise the software with respect to a given property. We evaluate our property-based mutation testing framework on Simulink models of safety-critical Cyber-Physical Systems (CPS) from the automotive and avionic domains and demonstrate how property-based mutation testing is more informative than regular mutation testing. These results open new perspectives in both mutation testing and test case generation of CPS.

翻译：变异测试是一种成熟的软件质量保证技术，用于评估测试套件的有效性。虽然该技术适用于预估测试套件的一般故障检测能力，但当被测软件必须针对特定需求进行验证时，它既不实用也缺乏信息量。这在嵌入式软件中尤为常见，此类软件通常需针对严格规范的安全属性进行验证。在此类场景中：(i) 仅当变异体能影响被测属性的满足程度时才具有相关性，(ii) 仅当变异体导致属性违反时，才被视为针对该属性有意义地消灭。为应对变异测试的这些局限性，我们提出属性基变异测试——一种评估测试套件在给定属性下执行软件能力的方法。我们在汽车和航空领域的安全关键信息物理系统的Simulink模型上评估了该框架，并展示了属性基变异测试相比常规变异测试能提供更丰富的信息。这些结果为信息物理系统的变异测试及测试用例生成开辟了新视角。