Reinforcement learning (RL) for bipedal locomotion has recently demonstrated robust gaits over moderate terrains using only proprioceptive sensing. However, such blind controllers will fail in environments where robots must anticipate and adapt to local terrain, which requires visual perception. In this paper, we propose a fully-learned system that allows bipedal robots to react to local terrain while maintaining commanded travel speed and direction. Our approach first trains a controller in simulation using a heightmap expressed in the robot's local frame. Next, data is collected in simulation to train a heightmap predictor, whose input is the history of depth images and robot states. We demonstrate that with appropriate domain randomization, this approach allows for successful sim-to-real transfer with no explicit pose estimation and no fine-tuning using real-world data. To the best of our knowledge, this is the first example of sim-to-real learning for vision-based bipedal locomotion over challenging terrains.

翻译：强化学习在双足运动领域近期已展现出仅利用本体感觉即可在中等地形上生成稳健步态的能力。然而，在机器人需预测并适应局部地形的环境中，此类无视觉控制器将失效，这要求引入视觉感知。本文提出一种全学习系统，使双足机器人既能响应局部地形变化，又能保持指令行进速度与方向。该方法首先利用机器人局部坐标系中的高度图在仿真中训练控制器，进而通过仿真采集数据训练高度图预测器，其输入为深度图像与机器人状态的历史序列。研究表明，通过恰当的域随机化，该方法无需显式位姿估计与真实数据微调即可实现成功的仿真到现实迁移。据我们所知，这是首个实现基于视觉的双足运动在复杂地形上的仿真到现实学习案例。