The simulation-to-reality (sim-to-real) transfer of large-scale hydraulic robots presents a significant challenge in robotics because of the inherent slow control response and complex fluid dynamics. The complex dynamics result from the multiple interconnected cylinder structure and the difference in fluid rates of the cylinders. These characteristics complicate detailed simulation for all joints, making it unsuitable for reinforcement learning (RL) applications. In this work, we propose an analytical actuator model driven by hydraulic dynamics to represent the complicated actuators. The model predicts joint torques for all 12 actuators in under 1 microsecond, allowing rapid processing in RL environments. We compare our model with neural network-based actuator models and demonstrate the advantages of our model in data-limited scenarios. The locomotion policy trained in RL with our model is deployed on a hydraulic quadruped robot, which is over 300 kg. This work is the first demonstration of a successful transfer of stable and robust command-tracking locomotion with RL on a heavy hydraulic quadruped robot, demonstrating advanced sim-to-real transferability.

翻译：大型液压机器人的仿真到现实（sim-to-real）迁移是机器人学中的重大挑战，这源于其固有的控制响应迟缓和复杂的流体动力学特性。复杂动力学由多缸互联结构及各缸流体速率差异导致。这些特性使得对所有关节进行精细仿真变得困难，从而不适用于强化学习（RL）应用。本研究提出一种基于液压动力学驱动的解析执行器模型，用以表征复杂的执行器系统。该模型可在1微秒内预测全部12个执行器的关节扭矩，满足强化学习环境对快速处理的需求。我们将本模型与基于神经网络的执行器模型进行比较，并展示了本模型在数据受限场景中的优势。使用本模型在强化学习中训练的运动策略已部署于重量超过300公斤的液压四足机器人上。此项工作首次实现了基于强化学习的稳定、鲁棒指令跟踪运动在重型液压四足机器人上的成功迁移，展现了先进的仿真到现实迁移能力。