As robots acquire increasingly sophisticated skills and see increasingly complex and varied environments, the threat of an edge case or anomalous failure is ever present. For example, Tesla cars have seen interesting failure modes ranging from autopilot disengagements due to inactive traffic lights carried by trucks to phantom braking caused by images of stop signs on roadside billboards. These system-level failures are not due to failures of any individual component of the autonomy stack but rather system-level deficiencies in semantic reasoning. Such edge cases, which we call \textit{semantic anomalies}, are simple for a human to disentangle yet require insightful reasoning. To this end, we study the application of large language models (LLMs), endowed with broad contextual understanding and reasoning capabilities, to recognize these edge semantic cases. We introduce a monitoring framework for semantic anomaly detection in vision-based policies to do so. Our experiments evaluate this framework in monitoring a learned policy for object manipulation and a finite state machine policy for autonomous driving and demonstrate that an LLM-based monitor can serve as a proxy for human reasoning. Finally, we provide an extended discussion on the strengths and weaknesses of this approach and motivate a research outlook on how we can further use foundation models for semantic anomaly detection.

翻译：随着机器人逐渐掌握更复杂的技能并面对日益复杂多变的环境，边缘情况或异常故障的威胁始终存在。例如，特斯拉汽车曾出现有趣的故障模式，包括因卡车运载的未激活交通灯导致自动驾驶系统退出，以及因路边广告牌上的停车标志图像引发幻影制动。这些系统级故障并非源于自主技术栈中任何单一组件的失效，而是语义推理层面的系统级缺陷。此类边缘情况——我们称之为"语义异常"——对人类而言易于辨明，却需要深刻的推理能力。为此，我们研究如何利用具备广泛语境理解与推理能力的大型语言模型（LLMs）来识别这些语义边缘情况。我们引入一个基于视觉策略的监控框架，用于实现语义异常检测。实验评估了该框架在监控物体操作学习策略与自动驾驶有限状态机策略中的表现，结果表明基于LLM的监控器可作为人类推理的有效替代。最后，我们深入探讨了该方法的优势与局限，并提出了利用基础模型进一步开展语义异常检测的研究展望。