We study the problem of mobile manipulation using legged robots equipped with an arm, namely legged loco-manipulation. The robot legs, while usually utilized for mobility, offer an opportunity to amplify the manipulation capabilities by conducting whole-body control. That is, the robot can control the legs and the arm at the same time to extend its workspace. We propose a framework that can conduct the whole-body control autonomously with visual observations. Our approach, namely Visual Whole-Body Control(VBC), is composed of a low-level policy using all degrees of freedom to track the end-effector manipulator position and a high-level policy proposing the end-effector position based on visual inputs. We train both levels of policies in simulation and perform Sim2Real transfer for real robot deployment. We perform extensive experiments and show significant improvements over baselines in picking up diverse objects in different configurations (heights, locations, orientations) and environments. Project page: https://wholebody-b1.github.io

翻译：我们研究使用配备机械臂的四足机器人进行移动操作的问题，即四足移动操作。机器人腿部通常用于移动，但通过实施全身控制，可增强操作能力。这意味着机器人能够同时控制腿部和手臂以扩展其工作空间。我们提出一种基于视觉观测自主执行全身控制的框架。该方法名为视觉全身控制(VBC)，包含两级策略：底层策略利用所有自由度追踪末端执行器位置，高层策略基于视觉输入提出末端执行器目标位置。我们在仿真环境中训练两级策略，并通过Sim2Real迁移实现真实机器人部署。大量实验表明，本方法在拾取不同配置（高度、位置、方向）和环境中的多样化物体方面，相较于基线方法取得显著改进。项目主页：https://wholebody-b1.github.io