The ability to detect and analyze failed executions automatically is crucial for an explainable and robust robotic system. Recently, Large Language Models (LLMs) have demonstrated strong reasoning abilities on textual inputs. To leverage the power of LLM for robot failure explanation, we introduce a framework REFLECT, which queries LLM to identify and explain robot failures given a hierarchical summary of robot past experiences generated from multi-sensory data. Conditioned on the explanation, a task planner will generate an executable plan for the robot to correct the failure and complete the task. To systematically evaluate the framework, we create the RoboFail dataset with a variety of tasks and failure scenarios. We demonstrate that the LLM-based framework is able to generate informative failure explanations that assist successful correction planning. Videos and code available at: https://roboreflect.github.io/.

翻译：自动检测和分析失败执行的能力对可解释且鲁棒的机器人系统至关重要。最近，大语言模型（LLMs）在文本输入上展示了强大的推理能力。为利用大语言模型进行机器人失败解释，我们提出了REFLECT框架，该框架查询大语言模型，基于由多感官数据生成的机器人过去经验的层次化总结来识别和解释机器人失败。基于该解释，任务规划器将生成可执行计划，使机器人纠正失败并完成任务。为系统评估该框架，我们创建了包含多种任务和失败场景的RoboFail数据集。实验证明，基于大语言模型的框架能够生成信息丰富的失败解释，从而辅助成功的纠正规划。视频和代码见：https://roboreflect.github.io/。