Recent years have witnessed many successful trials in the robot learning field. For contact-rich robotic tasks, it is challenging to learn coordinated motor skills by reinforcement learning. Imitation learning solves this problem by using a mimic reward to encourage the robot to track a given reference trajectory. However, imitation learning is not so efficient and may constrain the learned motion. In this paper, we propose instruction learning, which is inspired by the human learning process and is highly efficient, flexible, and versatile for robot motion learning. Instead of using a reference signal in the reward, instruction learning applies a reference signal directly as a feedforward action, and it is combined with a feedback action learned by reinforcement learning to control the robot. Besides, we propose the action bounding technique and remove the mimic reward, which is shown to be crucial for efficient and flexible learning. We compare the performance of instruction learning with imitation learning, indicating that instruction learning can greatly speed up the training process and guarantee learning the desired motion correctly. The effectiveness of instruction learning is validated through a bunch of motion learning examples for a biped robot and a quadruped robot, where skills can be learned typically within several million steps. Besides, we also conduct sim-to-real transfer and online learning experiments on a real quadruped robot. Instruction learning has shown great merits and potential, making it a promising alternative for imitation learning.

翻译：近年来，机器人学习领域涌现出许多成功案例。对于接触密集的机器人任务而言，通过强化学习掌握协调的运动技能具有挑战性。模仿学习通过使用模仿奖励来鼓励机器人跟踪给定的参考轨迹，从而解决了这一问题。然而，模仿学习效率较低，并且可能限制习得运动的多样性。在本文中，我们提出指令学习方法，该方法受人类学习过程启发，在机器人运动学习中具有高效性、灵活性和通用性。与在奖励中使用参考信号不同，指令学习将参考信号直接作为前馈动作应用，并与通过强化学习学得的反馈动作相结合来控制机器人。此外，我们提出了动作边界技术并去除了模仿奖励，这对高效灵活的学习至关重要。我们对比了指令学习与模仿学习的性能，结果表明指令学习能够极大加速训练过程，并确保正确习得期望动作。通过双足机器人和四足机器人的一系列运动学习实例验证了指令学习的有效性，其技能通常可在几百万步内习得。此外，我们还对真实四足机器人进行了仿真到现实迁移和在线学习实验。指令学习展现出显著优势与潜力，成为模仿学习的有力替代方案。