Controlling contact forces during interactions is critical for locomotion and manipulation tasks. While sim-to-real reinforcement learning (RL) has succeeded in many contact-rich problems, current RL methods achieve forceful interactions implicitly without explicitly regulating forces. We propose a method for training RL policies for direct force control without requiring access to force sensing. We showcase our method on a whole-body control platform of a quadruped robot with an arm. Such force control enables us to perform gravity compensation and impedance control, unlocking compliant whole-body manipulation. The learned whole-body controller with variable compliance makes it intuitive for humans to teleoperate the robot by only commanding the manipulator, and the robot's body adjusts automatically to achieve the desired position and force. Consequently, a human teleoperator can easily demonstrate a wide variety of loco-manipulation tasks. To the best of our knowledge, we provide the first deployment of learned whole-body force control in legged manipulators, paving the way for more versatile and adaptable legged robots.

翻译：在交互过程中控制接触力对于运动与操作任务至关重要。虽然仿真到现实的强化学习已在许多接触密集型问题中取得成功，但当前强化学习方法通过隐式方式实现有力交互，并未显式调节力。我们提出了一种无需力传感即可直接进行力控制的强化学习策略训练方法。我们在带机械臂的四足机器人全身控制平台上展示了该方法。这种力控使我们能够实现重力补偿和阻抗控制，从而解锁柔顺的全身操作。具有可变柔顺性的学习型全身控制器使操作人员仅通过操控机械臂即可直观地远程操控机器人，而机器人的身体会自动调整以实现期望的位置和力。因此，人类操作员可以轻松完成多种移动操作任务的演示。据我们所知，我们首次实现了腿部操作器中学习型全身力控制的部署，为更具通用性和适应性的腿部机器人奠定了基础。