Current approaches to humanoid control generally fall into two paradigms: perceptive locomotion, which handles terrain well but is limited to pedal gaits, and general motion tracking, which reproduces complex skills but ignores environmental capabilities. This work unites these paradigms to achieve perceptive general motion control. We present a framework where exteroceptive sensing is integrated into whole-body motion tracking, permitting a humanoid to perform highly dynamic, non-locomotion tasks on uneven terrain. By training a single policy to perform multiple distinct motions across varied terrestrial features, we demonstrate the non-trivial benefit of integrating perception into the control loop. Our results show that this framework enables robust, highly dynamic multi-contact motions, such as vaulting and dive-rolling, on unstructured terrain, significantly expanding the robot's traversability beyond simple walking or running. https://project-instinct.github.io/deep-whole-body-parkour

翻译：当前的人形机器人控制方法主要分为两种范式：感知式移动，其能良好处理地形但仅限于足部步态；以及通用运动跟踪，其能复现复杂技能但忽视环境适应能力。本研究融合这两种范式以实现感知式通用运动控制。我们提出一个将外感受感知整合至全身运动跟踪的框架，使人形机器人能够在非平坦地形上执行高度动态的非移动任务。通过训练单一策略在多样化地形特征上执行多个不同动作，我们证明了将感知整合至控制回路的重要优势。实验结果表明，该框架能够在非结构化地形上实现鲁棒且高度动态的多接触运动，例如撑跳和翻滚前扑，显著扩展了机器人超越简单行走或奔跑的穿越能力。https://project-instinct.github.io/deep-whole-body-parkour