This paper introduces a new parallel run-time for QuickCheck, a Haskell library and EDSL for specifying and randomly testing properties of programs. The new run-time can run multiple tests for a single property in parallel, using the available cores. Moreover, if a counterexample is found, the run-time can also shrink the test case in parallel, implementing a parallel search for a locally minimal counterexample. Our experimental results show a 3--9$\times$ speed-up for testing QuickCheck properties on a variety of heavy-weight benchmark problems. We also evaluate two different shrinking strategies; deterministic shrinking, which guarantees to produce the same minimal test case as standard sequential shrinking, and greedy shrinking, which does not have this guarantee but still produces a locally minimal test case, and is faster in practice.
翻译:本文介绍了一种为QuickCheck(一种用于指定和随机测试程序属性的Haskell库与EDSL)设计的新型并行运行时。该运行时可利用多核并行执行同一属性的多组测试。此外,当发现反例时,运行时还能并行进行测试用例缩减,实现局部最小反例的并行搜索。实验结果表明,在多种重量级基准问题上,该并行运行时对QuickCheck属性的测试实现了3-9倍的加速。我们还评估了两种不同的缩减策略:确定性缩减(保证产生与标准顺序缩减相同的最小测试用例)与贪婪性缩减(虽无此保证但仍能产生局部最小测试用例,且在实际运行中速度更快)。