The ability to detect and analyze failed executions automatically is crucial for an explainable and robust robotic system. Recently, Large Language Models (LLMs) have demonstrated strong common sense reasoning skills on textual inputs. To leverage the power of LLM for robot failure explanation, we propose a framework REFLECT, which converts multi-sensory data into a hierarchical summary of robot past experiences and queries LLM with a progressive failure explanation algorithm. Conditioned on the explanation, a failure correction planner generates an executable plan for the robot to correct the failure and complete the task. To systematically evaluate the framework, we create the RoboFail dataset and show that our LLM-based framework is able to generate informative failure explanations that assist successful correction planning. Project website: https://roboreflect.github.io/

翻译：自动检测并分析失败执行的能力对于构建可解释且鲁棒的机器人系统至关重要。近年来，大型语言模型（LLMs）在文本输入上展现了强大的常识推理能力。为利用LLM进行机器人失败解释，我们提出REFLECT框架，该框架将多感官数据转换为机器人过往经验的层次化摘要，并通过渐进式失败解释算法查询LLM。基于解释结果，一个失败纠正规划器会生成可执行计划，以帮助机器人修正失败并完成任务。为系统评估该框架，我们构建了RoboFail数据集，并表明基于LLM的框架能够生成信息丰富的失败解释，从而辅助成功的纠正规划。项目网站：https://roboreflect.github.io/