Small Unmanned Aerial Systems (sUAS) must meet rigorous safety standards when deployed in high-stress emergency response scenarios. However, tests that execute perfectly in simulation can fail dramatically in real-world environments. Fuzz testing can be used to increase system robustness by providing malformed input data aimed at triggering failure cases. In this paper, we apply fuzzing to support human interaction testing. Initial tests are run in simulation to provide broad coverage of the input space in a safe environment; however, they lack the fidelity of real-world tests. Field tests provide higher fidelity but can result in costly or dangerous crashes. We, therefore, propose and demonstrate HiFuzz, which executes large numbers of fuzz tests in simulation and then down-selects tests for deployment in human-in-the-loop simulations and safety-aware physical field tests. We apply \hf to a multi-sUAS system and show that each test level serves a unique purpose in identifying known and unknown failures associated with human interactions.
翻译:摘要:小型无人驾驶航空系统(sUAS)在部署于高压应急响应场景时必须满足严格的安全标准。然而,在仿真环境中完美执行的测试在真实环境中可能会严重失败。模糊测试通过提供旨在触发故障案例的畸形输入数据,可用于增强系统鲁棒性。本文提出将模糊测试应用于支持人机交互测试。初始测试在仿真环境中运行,以在安全条件下实现对输入空间的广泛覆盖,但缺乏真实环境测试的保真度。现场测试虽具有更高保真度,却可能导致昂贵或危险的坠毁事件。为此,我们提出并验证了HiFuzz方法:该方法在仿真环境中执行大量模糊测试,随后筛选出适用于人在回路仿真及安全感知物理现场测试的测试用例。我们将HiFuzz应用于一个多sUAS系统,结果表明每个测试层级在识别与人机交互相关的已知及未知故障中均发挥着独特作用。