Elevator systems are one kind of Cyber-Physical Systems (CPSs), and as such, test cases are usually complex and long in time. This is mainly because realistic test scenarios are employed (e.g., for testing elevator dispatching algorithms, typically a full day of passengers traveling through a system of elevators is used). However, in such a context, when needing to reproduce a failure, it is of high benefit to provide the minimal test input to the software developers. This way, analyzing and trying to localize the root-cause of the failure is easier and more agile. Delta debugging has been found to be an efficient technique to reduce failure-inducing test inputs. In this paper, we enhance this technique by first monitoring the environment at which the CPS operates as well as its physical states. With the monitored information, we search for stable states of the CPS during the execution of the simulation. In a second step, we use such identified stable states to help the delta debugging algorithm isolate the failure-inducing test inputs more efficiently. We report our experience of applying our approach into an industrial elevator dispatching algorithm. An empirical evaluation carried out with real operational data from a real installation of elevators suggests that the proposed environment-wise delta debugging algorithm is between 1.3 to 1.8 times faster than the traditional delta debugging, while producing a larger reduction in the failure-inducing test inputs. The results provided by the different implemented delta debugging algorithm versions are qualitatively assessed with domain experts. This assessment provides new insights and lessons learned, such as, potential applications of the delta debugging algorithm beyond debugging.
翻译:电梯系统是信息物理系统(CPS)的一种,其测试用例通常复杂且耗时较长。这主要是因为采用真实测试场景(例如,测试电梯调度算法时,通常使用一整天的乘客在电梯系统中运行的数据)。然而,在此背景下,当需要复现故障时,向软件开发者提供最小化测试输入具有显著优势。这样,分析和定位故障根本原因将更加便捷高效。Delta调试已被证明是一种能够有效缩减诱发故障的测试输入的技术。本文通过首先监控CPS运行的环境及其物理状态,对该技术进行了改进。利用监控信息,我们搜索仿真执行过程中CPS的稳定状态。第二步,利用这些识别出的稳定状态,帮助Delta调试算法更高效地隔离诱发故障的测试输入。我们报告了将所提方法应用于工业电梯调度算法的经验。基于真实电梯安装现场的运行数据进行的实证评估表明,所提出的环境感知型Delta调试算法比传统Delta调试算法快1.3至1.8倍,同时能更大程度地缩减诱发故障的测试输入。不同版本的Delta调试算法所产生的结果由领域专家进行定性评估。这一评估提供了新的见解和经验教训,例如Delta调试算法在调试之外的潜在应用。