Machine learning driven image-based controllers allow robotic systems to take intelligent actions based on the visual feedback from their environment. Understanding when these controllers might lead to system safety violations is important for their integration in safety-critical applications and engineering corrective safety measures for the system. Existing methods leverage simulation-based testing (or falsification) to find the failures of vision-based controllers, i.e., the visual inputs that lead to closed-loop safety violations. However, these techniques do not scale well to the scenarios involving high-dimensional and complex visual inputs, such as RGB images. In this work, we cast the problem of finding closed-loop vision failures as a Hamilton-Jacobi (HJ) reachability problem. Our approach blends simulation-based analysis with HJ reachability methods to compute an approximation of the backward reachable tube (BRT) of the system, i.e., the set of unsafe states for the system under vision-based controllers. Utilizing the BRT, we can tractably and systematically find the system states and corresponding visual inputs that lead to closed-loop failures. These visual inputs can be subsequently analyzed to find the input characteristics that might have caused the failure. Besides its scalability to high-dimensional visual inputs, an explicit computation of BRT allows the proposed approach to capture non-trivial system failures that are difficult to expose via random simulations. We demonstrate our framework on two case studies involving an RGB image-based neural network controller for (a) autonomous indoor navigation, and (b) autonomous aircraft taxiing.

翻译：机器学习驱动的基于图像的控制器使机器人系统能够根据环境的视觉反馈做出智能决策。了解这些控制器何时可能导致系统违反安全规范，对于将其集成到安全关键应用以及制定工程化的系统安全纠正措施至关重要。现有方法利用基于模拟的测试（或反证法）来发现视觉控制器的失效，即导致闭环安全违规的视觉输入。然而，这些技术在处理涉及高维复杂视觉输入（如RGB图像）的场景时扩展性不佳。本研究将寻找闭环视觉失效问题建模为Hamilton-Jacobi (HJ) 可达性问题。我们的方法融合了基于模拟的分析与HJ可达性方法，以计算系统后向可达管（BRT）的近似值，即基于视觉控制器的系统不安全状态集合。利用BRT，我们能够可计算且系统地找到导致闭环失效的系统状态及相应视觉输入。随后可分析这些视觉输入，以确定可能引发失效的输入特征。除了对高维视觉输入的可扩展性外，BRT的显式计算使所提出的方法能够捕获通过随机模拟难以暴露的非平凡系统失效。我们通过两个案例研究验证了该框架，涉及基于RGB图像的神经网络控制器，分别用于(a)自主室内导航和(b)自主飞机滑行。