Autonomous systems, such as self-driving cars and drones, have made significant strides in recent years by leveraging visual inputs and machine learning for decision-making and control. Despite their impressive performance, these vision-based controllers can make erroneous predictions when faced with novel or out-of-distribution inputs. Such errors can cascade to catastrophic system failures and compromise system safety. In this work, we introduce a run-time anomaly monitor to detect and mitigate such closed-loop, system-level failures. Specifically, we leverage a reachability-based framework to stress-test the vision-based controller offline and mine its system-level failures. This data is then used to train a classifier that is leveraged online to flag inputs that might cause system breakdowns. The anomaly detector highlights issues that transcend individual modules and pertain to the safety of the overall system. We also design a fallback controller that robustly handles these detected anomalies to preserve system safety. We validate the proposed approach on an autonomous aircraft taxiing system that uses a vision-based controller for taxiing. Our results show the efficacy of the proposed approach in identifying and handling system-level anomalies, outperforming methods such as prediction error-based detection, and ensembling, thereby enhancing the overall safety and robustness of autonomous systems.

翻译：自主系统（如自动驾驶汽车和无人机）近年来通过利用视觉输入和机器学习进行决策与控制取得了显著进展。尽管这些基于视觉的控制器表现出色，但在面对新型或分布外输入时可能做出错误预测。此类错误可能级联为灾难性系统故障，危及系统安全性。本文提出一种运行时异常监控器，用于检测和缓解此类闭环系统级故障。具体而言，我们利用基于可达性的框架对基于视觉的控制器进行离线压力测试，挖掘其系统级故障。随后利用该数据训练分类器，在线标记可能导致系统崩溃的输入。该异常检测器可识别超越单个模块、关乎整体系统安全性的问题。同时设计了一个回退控制器，可稳健处理检测到的异常以维护系统安全。我们在基于视觉控制器的自主飞机滑行系统中验证了所提方法。结果表明，该方法能有效识别和处理系统级异常，优于基于预测误差的检测和集成方法，从而增强了自主系统的整体安全性与鲁棒性。