Metamorphic testing alleviates the test oracle problem by defining oracles based on relations (i.e., metamorphic relations) that govern multiple related inputs and their expected outputs. However, designing MRs requires domain-specific knowledge, which may not be generally available. This hinders the widespread adoption of MT. Fortunately, we observe that developers often embed such domain knowledge that encodes MRs in test cases, even without the awareness of metamorphic testing. These encoded MRs might be generalized and useful for testing the original programs (where these MRs are found) and other programs with similar functionalities. In this paper, we propose MR-Extractor, to automatically discover metamorphic relations from existing test cases in open-source software (OSS) projects. MR-Extractor firstly identifies MR-encoded test cases in OSS, and then generalizes encoded MRs and codifies them into parameterized methods (called codified MR) for MT. Finally, MR-Extractor validates the reusability of codified MRs for new test generation. Experimental results show that (i) over 12,000 MR-encoded test cases are identified by MR-Extractor, accounting for 1.26% of the total test cases, and (ii) test cases constructed from the MRs discovered by MR-Extractor can effectively improve test coverage. Compared with developer-written test suites, codified-MR-based test suites leads to a 25.97% and 25.83% increase in the line coverage and mutation score. When compared with Evosuite-generated test suites, codified-MR-based test suites even achieves an 83.11% increase in mutant score.
翻译:蜕变测试通过定义基于关系的预言(即蜕变关系)来缓解测试预言问题,这类关系规定了多个关联输入及其预期输出。然而,设计蜕变关系需要特定领域的知识,而这些知识通常并非普遍可得。这阻碍了蜕变测试的广泛采用。幸运的是,我们发现开发者即使未意识到蜕变测试,也常常将编码了蜕变关系的领域知识嵌入测试用例中。这些被编码的蜕变关系可能具有通用性,不仅有助于测试原始程序(即发现这些蜕变关系的程序),也能用于测试具有类似功能的其他程序。本文提出MR-Extractor,用于从开源软件项目的现有测试用例中自动发现蜕变关系。MR-Extractor首先识别开源软件中编码了蜕变关系的测试用例,然后泛化这些编码的蜕变关系,并将其形式化为参数化方法(称为编码蜕变关系)以供蜕变测试使用。最后,MR-Extractor验证编码蜕变关系在生成新测试用例时的可重用性。实验结果表明:(i)MR-Extractor识别出超过12,000个编码了蜕变关系的测试用例,占测试用例总数的1.26%;(ii)基于MR-Extractor发现的蜕变关系构建的测试用例能有效提升测试覆盖率。与开发者编写的测试套件相比,基于编码蜕变关系的测试套件在行覆盖率和变异得分上分别提升了25.97%和25.83%;与Evosuite生成的测试套件相比,基于编码蜕变关系的测试套件在变异得分上甚至实现了83.11%的提升。