We study the problem of mobile manipulation using legged robots equipped with an arm, namely legged loco-manipulation. The robot legs, while usually utilized for mobility, offer an opportunity to amplify the manipulation capabilities by conducting whole-body control. That is, the robot can control the legs and the arm at the same time to extend its workspace. We propose a framework that can conduct the whole-body control autonomously with visual observations. Our approach, namely Visual Whole-Body Control(VBC), is composed of a low-level policy using all degrees of freedom to track the body velocities along with the end-effector position, and a high-level policy proposing the velocities and end-effector position based on visual inputs. We train both levels of policies in simulation and perform Sim2Real transfer for real robot deployment. We perform extensive experiments and show significant improvements over baselines in picking up diverse objects in different configurations (heights, locations, orientations) and environments.

翻译：我们研究了使用配备机械臂的腿式机器人进行移动操作的问题，即腿式移动操作。机器人的腿部通常用于移动，但通过实施全身控制，它们为增强操作能力提供了机会。也就是说，机器人可以同时控制腿部和机械臂以扩展其工作空间。我们提出了一个能够基于视觉观测自主实施全身控制的框架。我们的方法，即视觉全身控制（VBC），由一个使用所有自由度来跟踪身体速度及末端执行器位置的低层策略，以及一个基于视觉输入提出速度及末端执行器位置建议的高层策略组成。我们在仿真中训练这两个层级的策略，并执行Sim2Real迁移以部署到真实机器人上。我们进行了大量实验，结果表明，在不同配置（高度、位置、朝向）和环境中拾取多样化物体方面，本方法相较于基线有显著提升。