As robots acquire increasingly sophisticated skills and see increasingly complex and varied environments, the threat of an edge case or anomalous failure is ever present. For example, Tesla cars have seen interesting failure modes ranging from autopilot disengagements due to inactive traffic lights carried by trucks to phantom braking caused by images of stop signs on roadside billboards. These system-level failures are not due to failures of any individual component of the autonomy stack but rather system-level deficiencies in semantic reasoning. Such edge cases, which we call semantic anomalies, are simple for a human to disentangle yet require insightful reasoning. To this end, we study the application of large language models (LLMs), endowed with broad contextual understanding and reasoning capabilities, to recognize such edge cases and introduce a monitoring framework for semantic anomaly detection in vision-based policies. Our experiments apply this framework to a finite state machine policy for autonomous driving and a learned policy for object manipulation. These experiments demonstrate that the LLM-based monitor can effectively identify semantic anomalies in a manner that shows agreement with human reasoning. Finally, we provide an extended discussion on the strengths and weaknesses of this approach and motivate a research outlook on how we can further use foundation models for semantic anomaly detection.

翻译：随着机器人获得日益复杂的技能并面临日益复杂多变的环境，边缘案例或异常故障的威胁始终存在。例如，特斯拉汽车曾出现有趣的故障模式：从因卡车上装载的不慎激活的交通信号灯导致自动驾驶系统解除，到因路边广告牌上的停车标志图像引发幽灵刹车。这些系统级故障并非源于自主堆栈中任何单个组件的失效，而是语义推理层面的系统性缺陷。此类边缘案例（我们称之为语义异常）对人类而言易于辨析，却需要深刻的推理能力。为此，我们研究如何利用具备广泛语境理解与推理能力的大语言模型来识别此类边缘案例，并提出一个基于视觉策略的语义异常检测监控框架。实验将该框架应用于自动驾驶的有限状态机策略与物体操作的习得策略。结果表明，基于大语言模型的监控器能够有效识别语义异常，且其判断与人类推理结果高度一致。最后，我们深入探讨了该方法的优缺点，并展望了如何进一步利用基础模型进行语义异常检测的研究方向。