When cast into the Deep Reinforcement Learning framework, many robotics tasks require solving a long horizon and sparse reward problem, where learning algorithms struggle. In such context, Imitation Learning (IL) can be a powerful approach to bootstrap the learning process. However, most IL methods require several expert demonstrations which can be prohibitively difficult to acquire. Only a handful of IL algorithms have shown efficiency in the context of an extreme low expert data regime where a single expert demonstration is available. In this paper, we present a novel algorithm designed to imitate complex robotic tasks from the states of an expert trajectory. Based on a sequential inductive bias, our method divides the complex task into smaller skills. The skills are learned into a goal-conditioned policy that is able to solve each skill individually and chain skills to solve the entire task. We show that our method imitates a non-holonomic navigation task and scales to a complex simulated robotic manipulation task with very high sample efficiency.

翻译：当纳入深度强化学习框架时，许多机器人任务需要解决长时域和稀疏奖励问题，而学习算法在此类问题中面临挑战。在此背景下，模仿学习（Imitation Learning, IL）可以成为引导学习过程的强大方法。然而，大多数模仿学习方法需要多次专家演示，这在实际获取中可能极具难度。仅有少数模仿学习算法在极端低专家数据场景（仅有一次专家演示可用）下展现出有效性。本文提出一种新颖算法，旨在从专家轨迹的状态中模仿复杂机器人任务。基于序列归纳偏置，该方法将复杂任务分解为更小的技能。这些技能被学习为一个以目标为条件的策略，该策略能单独解决每个技能，并通过技能串联解决整个任务。实验表明，我们的方法能以极高的样本效率模仿非完整约束导航任务，并扩展到复杂的模拟机器人操作任务。