In partially known environments, robots must combine exploration to gather information with task planning for efficient execution. To address this challenge, we propose EPoG, an Exploration-based sequential manipulation Planning framework on Scene Graphs. EPoG integrates a graph-based global planner with a Large Language Model (LLM)-based situated local planner, continuously updating a belief graph using observations and LLM predictions to represent known and unknown objects. Action sequences are generated by computing graph edit operations between the goal and belief graphs, ordered by temporal dependencies and movement costs. This approach seamlessly combines exploration and sequential manipulation planning. In ablation studies across 46 realistic household scenes and 5 long-horizon daily object transportation tasks, EPoG achieved a success rate of 91.3%, reducing travel distance by 36.1% on average. Furthermore, a physical mobile manipulator successfully executed complex tasks in unknown and dynamic environments, demonstrating EPoG's potential for real-world applications.
翻译:在部分已知环境中,机器人必须将信息收集的探索过程与任务规划相结合以实现高效执行。为应对这一挑战,我们提出EPoG——一种基于场景图的探索式序列化操作规划框架。EPoG将基于图的全局规划器与基于大语言模型的情境局部规划器相集成,通过持续利用观测数据和LLM预测结果更新信念图,以表征已知与未知物体。通过计算目标图与信念图之间的图编辑操作,并依据时序依赖关系与移动成本进行排序,系统生成动作序列。该方法实现了探索与序列化操作规划的无缝融合。在涵盖46个真实家居场景与5项长时程日常物体搬运任务的消融实验中,EPoG取得了91.3%的成功率,平均移动距离减少36.1%。此外,实体移动机械臂在未知动态环境中成功执行了复杂任务,证明了EPoG在实际应用中的潜力。