Legged locomotion is arguably the most suited and versatile mode to deal with natural or unstructured terrains. Intensive research into dynamic walking and running controllers has recently yielded great advances, both in the optimal control and reinforcement learning (RL) literature. Hopping is a challenging dynamic task involving a flight phase and has the potential to increase the traversability of legged robots. Model based control for hopping typically relies on accurate detection of different jump phases, such as lift-off or touch down, and using different controllers for each phase. In this paper, we present a end-to-end RL based torque controller that learns to implicitly detect the relevant jump phases, removing the need to provide manual heuristics for state detection. We also extend a method for simulation to reality transfer of the learned controller to contact rich dynamic tasks, resulting in successful deployment on the robot after training without parameter tuning.

翻译：有腿运动可以说是处理自然或非结构化地形最合适且最通用的方式。近期针对动态行走和跑步控制器的深入研究，在最优控制和强化学习（RL）领域均取得了重大进展。跳跃是一项涉及飞行阶段的动态挑战性任务，具有提升有腿机器人可穿越性的潜力。基于模型的跳跃控制通常依赖于对起飞、触地等不同跳跃阶段的精确检测，并为每个阶段使用不同的控制器。本文提出了一种基于端到端强化学习的扭矩控制器，该控制器能够隐式学习检测相关跳跃阶段，从而无需提供用于状态检测的人工启发式规则。我们还扩展了一种将学习到的控制器从仿真迁移到现实世界的方法，使其适用于接触丰富的动态任务，从而在无需参数调整的情况下成功将训练后的控制器部署到机器人上。