Metamorphic Testing (MT) is a testing technique that can effectively alleviate the oracle problem. MT uses Metamorphic Relations (MRs) to determine if a test case passes or fails. MRs specify how the outputs should vary in response to specific input changes when executing the System Under Test (SUT). If a particular MR is violated for at least one test input (and its change), there is a high probability that the SUT has a fault. On the other hand, if a particular MR is not violated, it does not guarantee that the SUT is fault free. However, deciding if the MR is being violated due to a bug or because the MR does not hold/fit for particular conditions generated by specific inputs remains a manual task and unexplored. In this paper, we develop a method for refining MRs to offer hints as to whether a violation results from a bug or arises from the MR not being matched to certain test data under specific circumstances. In our initial proof-of-concept, we derive the relevant information from rules using the Association Rule Mining (ARM) technique. In our initial proof-of-concept, we validate our method on a toy example and discuss the lessons learned from our experiments. Our proof-of-concept demonstrates that our method is applicable and that we can provide suggestions that help strengthen the test suite for regression testing purposes.
翻译:蜕变测试(MT)是一种能有效缓解Oracle问题的测试技术。MT利用蜕变关系(MRs)判定测试用例是否通过。蜕变关系规定了当系统执行被测试系统(SUT)时,输出应如何随特定输入变化而改变。若特定MR在至少一个测试输入(及其变化)上被违反,则SUT很可能存在缺陷。反之,若MR未被违反,并不能保证SUT无缺陷。然而,判断MR违反是由于程序缺陷还是因特定输入产生的条件导致MR不成立/不匹配,仍依赖人工操作且尚待探索。本文提出一种改进MR的方法,用于提示违反结果源于缺陷还是MR在特定环境下与测试数据不匹配。在初步概念验证中,我们采用关联规则挖掘(ARM)技术从规则中提取相关信息。通过一个玩具示例验证了方法有效性,并讨论了实验所得启示。该概念验证表明本方法具有可行性,能为回归测试提供加强测试套件的建议。