Autonomous systems, such as self-driving cars and drones, have made significant strides in recent years by leveraging visual inputs and machine learning for decision-making and control. Despite their impressive performance, these vision-based controllers can make erroneous predictions when faced with novel or out-of-distribution inputs. Such errors can cascade to catastrophic system failures and compromise system safety. In this work, we introduce a run-time anomaly monitor to detect and mitigate such closed-loop, system-level failures. Specifically, we leverage a reachability-based framework to stress-test the vision-based controller offline and mine its system-level failures. This data is then used to train a classifier that is leveraged online to flag inputs that might cause system breakdowns. The anomaly detector highlights issues that transcend individual modules and pertain to the safety of the overall system. We also design a fallback controller that robustly handles these detected anomalies to preserve system safety. We validate the proposed approach on an autonomous aircraft taxiing system that uses a vision-based controller for taxiing. Our results show the efficacy of the proposed approach in identifying and handling system-level anomalies, outperforming methods such as prediction error-based detection, and ensembling, thereby enhancing the overall safety and robustness of autonomous systems.

翻译：自主系统，如自动驾驶汽车和无人机，近年来通过利用视觉输入和机器学习进行决策与控制取得了显著进展。尽管这些基于视觉的控制器表现卓越，但面对新颖或分布外输入时可能产生错误预测。此类错误可能级联升级为灾难性系统故障，危及系统安全性。本文提出一种运行时异常监控器，用于检测并缓解此类闭环系统级故障。具体而言，我们利用基于可达性的框架对视觉控制器进行离线压力测试，挖掘其系统级故障。这些数据随后用于训练分类器，在线标记可能导致系统崩溃的输入。该异常检测器能揭示超越单个模块、关乎整体系统安全性的问题。我们还设计了一个鲁棒处理检测到异常的备用控制器，以维持系统安全。我们在使用基于视觉控制器进行滑行的自主飞机滑行系统上验证了所提方法。结果表明，该方法能有效识别与处理系统级异常，优于基于预测误差的检测和集成等方法，从而增强了自主系统的整体安全性与鲁棒性。