Is a manipulator on a legged robot a liability or an asset for locomotion? Prior works mainly designed specific controllers to account for the added payload and inertia from a manipulator. In contrast, biological systems typically benefit from additional limbs, which can simplify postural control. For instance, cats use their tails to enhance the stability of their bodies and prevent falls under disturbances. In this work, we show that a manipulator can be an important asset for maintaining balance during locomotion. To do so, we train a sensorimotor policy using deep reinforcement learning to create a synergy between the robot's limbs. This policy enables the robot to maintain stability despite large disturbances. However, learning such a controller can be quite challenging. To account for these challenges, we propose a stage-wise training procedure to learn complex behaviors. Our proposed method decomposes this complex task into three stages and then incrementally learns these tasks to arrive at a single policy capable of solving the final control task, achieving a success rate up to 2.35 times higher than baselines in simulation. We deploy our learned policy in the real world and show stability during locomotion under strong disturbances.

翻译：摘要：腿式机器人上的机械臂对于运动而言是负担还是资产？先前的研究主要通过设计特定控制器来应对机械臂带来的额外负载和惯量。相比之下，生物系统通常受益于额外的肢体，这些肢体能够简化姿态控制。例如，猫利用尾巴增强身体稳定性，并在受到干扰时防止摔倒。在本工作中，我们证明机械臂可以在运动过程中成为维持平衡的重要资产。为此，我们使用深度强化学习训练一个感知运动策略，以创建机器人肢体之间的协同作用。该策略使机器人即使在受到大幅度干扰时也能保持稳定性。然而，学习这样的控制器可能相当具有挑战性。为了应对这些挑战，我们提出了一种分阶段训练流程来学习复杂行为。我们的方法将这一复杂任务分解为三个阶段，并逐步学习这些任务，最终形成一个能够解决最终控制任务的单一策略，在仿真中成功率比基线高出2.35倍。我们将学到的策略部署到真实世界中，并展示了在强干扰下的运动稳定性。