Deep reinforcement learning (DRL) has emerged as an innovative solution for controlling legged robots in challenging environments using minimalist architectures. Traditional control methods for legged robots, such as inverse dynamics, either directly manage joint torques or use proportional-derivative (PD) controllers to regulate joint positions at a higher level. In case of DRL, direct torque control presents significant challenges, leading to a preference for joint position control. However, this approach necessitates careful adjustment of joint PD gains, which can limit both adaptability and efficiency. In this paper, we propose GainAdaptor, an adaptive gain control framework that autonomously tunes joint PD gains to enhance terrain adaptability and energy efficiency. The framework employs a dual-actor algorithm to dynamically adjust the PD gains based on varying ground conditions. By utilizing a divided action space, GainAdaptor efficiently learns stable and energy-efficient locomotion. We validate the effectiveness of the proposed method through experiments conducted on a Unitree Go1 robot, demonstrating improved locomotion performance across diverse terrains.

翻译：深度强化学习已成为利用简约架构在复杂环境中控制腿式机器人的创新解决方案。腿式机器人的传统控制方法，如逆动力学，要么直接管理关节扭矩，要么在更高层级使用比例-微分控制器调节关节位置。在深度强化学习中，直接扭矩控制面临显著挑战，因此关节位置控制更受青睐。然而，该方法需要对关节PD增益进行精细调节，这可能限制系统的适应性与能效。本文提出GainAdaptor——一种自适应增益控制框架，可自主调节关节PD增益以增强地形适应性与能量效率。该框架采用双执行器算法，根据地面的变化动态调整PD增益。通过利用分割的动作空间，GainAdaptor能够高效学习稳定且节能的运动模式。我们在Unitree Go1机器人上进行了实验验证，结果表明该方法在多种地形上均能提升运动性能。