Recent advances in hierarchical robot systems leverage a high-level planner to propose task plans and a low-level policy to generate robot actions. This design allows training the planner on action-free or even non-robot data sources (e.g., videos), providing transferable high-level guidance. Nevertheless, grounding these high-level plans into executable actions remains challenging, especially with the limited availability of high-quality robot data. To this end, we propose to improve the low-level policy through online interactions. Specifically, our approach collects online rollouts, retrospectively annotates the corresponding high-level goals from achieved outcomes, and aggregates these hindsight-relabeled experiences to update a goal-conditioned imitation policy. Our method, Hindsight Flow-conditioned Online Imitation (HinFlow), instantiates this idea with 2D point flows as the high-level planner. Across diverse manipulation tasks in both simulation and physical world, our method achieves more than $2\times$ performance improvement over the base policy, significantly outperforming the existing methods. Moreover, our framework enables policy acquisition from planners trained on cross-embodiment video data, demonstrating its potential for scalable and transferable robot learning.

翻译：近期分层机器人系统的进展利用高层规划器提出任务计划，并利用低层策略生成机器人动作。这种设计允许在无需动作数据甚至非机器人数据源（如视频）上训练规划器，从而提供可迁移的高层指导。然而，将这些高层计划落实到可执行动作仍然具有挑战性，尤其是在高质量机器人数据有限的情况下。为此，我们提出通过在线交互改进低层策略。具体而言，我们的方法收集在线轨迹，根据已实现的结果回顾性地标注对应的高层目标，并聚合这些经过后见之明重标注的经验以更新目标条件模仿策略。我们的方法——后见之明流条件在线模仿（HinFlow）——以二维点流作为高层规划器实现了这一思想。在仿真和物理世界的多种操作任务中，我们的方法相比基础策略实现了超过 $2\times$ 的性能提升，显著优于现有方法。此外，我们的框架支持从跨具身视频数据训练的规划器获取策略，展示了其在可扩展和可迁移机器人学习方面的潜力。