To assist with everyday human activities, robots must solve complex long-horizon tasks and generalize to new settings. Recent deep reinforcement learning (RL) methods show promise in fully autonomous learning, but they struggle to reach long-term goals in large environments. On the other hand, Task and Motion Planning (TAMP) approaches excel at solving and generalizing across long-horizon tasks, thanks to their powerful state and action abstractions. But they assume predefined skill sets, which limits their real-world applications. In this work, we combine the benefits of these two paradigms and propose an integrated task planning and skill learning framework named LEAGUE (Learning and Abstraction with Guidance). LEAGUE leverages the symbolic interface of a task planner to guide RL-based skill learning and creates abstract state space to enable skill reuse. More importantly, LEAGUE learns manipulation skills in-situ of the task planning system, continuously growing its capability and the set of tasks that it can solve. We evaluate LEAGUE on four challenging simulated task domains and show that LEAGUE outperforms baselines by large margins. We also show that the learned skills can be reused to accelerate learning in new tasks domains and transfer to a physical robot platform.

翻译：为辅助人类日常活动，机器人需解决复杂的长时程任务并泛化至新场景。最近的深度强化学习方法在全自主学习方面展现出潜力，但在大型环境中实现长期目标仍存在困难。另一方面，任务与运动规划方法凭借其强大的状态与动作抽象能力，在长时程任务的求解与泛化方面表现优异，但其预设技能集的特性限制了实际应用。本研究融合两种范式的优势，提出名为LEAGUE（引导式学习与抽象）的集成式任务规划与技能学习框架。LEAGUE利用任务规划器的符号接口引导基于强化学习的技能学习，并通过构建抽象状态空间实现技能复用。更重要的是，LEAGUE在任务规划系统原位环境下学习操作技能，持续增强自身能力及可解任务集合。我们在四个具有挑战性的模拟任务领域上评估LEAGUE，结果表明其性能大幅超越基线方法。同时验证了习得技能可迁移至新任务领域以加速学习，并成功部署至实体机器人平台。