Controlling contact forces during interactions is critical for locomotion and manipulation tasks. While sim-to-real reinforcement learning (RL) has succeeded in many contact-rich problems, current RL methods achieve forceful interactions implicitly without explicitly regulating forces. We propose a method for training RL policies for direct force control without requiring access to force sensing. We showcase our method on a whole-body control platform of a quadruped robot with an arm. Such force control enables us to perform gravity compensation and impedance control, unlocking compliant whole-body manipulation. The learned whole-body controller with variable compliance makes it intuitive for humans to teleoperate the robot by only commanding the manipulator, and the robot's body adjusts automatically to achieve the desired position and force. Consequently, a human teleoperator can easily demonstrate a wide variety of loco-manipulation tasks. To the best of our knowledge, we provide the first deployment of learned whole-body force control in legged manipulators, paving the way for more versatile and adaptable legged robots.

翻译：在交互过程中控制接触力对于移动和操作任务至关重要。尽管从仿真到现实的强化学习已在许多高接触性问题中取得成功，但当前的强化学习方法仅能隐式地实现强交互，而缺乏显式的力调节机制。我们提出了一种无需力传感即可训练直接力控制强化学习策略的方法。该方法在一台配备机械臂的四足机器人全身控制平台上进行了验证。这种力控制能力使我们能够实现重力补偿和阻抗控制，从而解锁柔顺的全身操作。具有可变柔顺性的全身控制策略使得人类操作员仅需操控机械臂即可直观地远程操控机器人，而机器人身体会自动调整以达成期望的位置和力。由此，人类操作员能够轻松演示多种多样的移动-操作任务。据我们所知，这是首次在腿式操作器上实现学习型全身力控制，为开发更通用、更适应性的腿式机器人铺平了道路。