Robot motor skills can be learned through deep reinforcement learning (DRL) by neural networks as state-action mappings. While the selection of state observations is crucial, there has been a lack of quantitative analysis to date. Here, we present a systematic saliency analysis that quantitatively evaluates the relative importance of different feedback states for motor skills learned through DRL. Our approach can identify the most essential feedback states for locomotion skills, including balance recovery, trotting, bounding, pacing and galloping. By using only key states including joint positions, gravity vector, base linear and angular velocities, we demonstrate that a simulated quadruped robot can achieve robust performance in various test scenarios across these distinct skills. The benchmarks using task performance metrics show that locomotion skills learned with key states can achieve comparable performance to those with all states, and the task performance or learning success rate will drop significantly if key states are missing. This work provides quantitative insights into the relationship between state observations and specific types of motor skills, serving as a guideline for robot motor learning. The proposed method is applicable to differentiable state-action mapping, such as neural network based control policies, enabling the learning of a wide range of motor skills with minimal sensing dependencies.

翻译：机器人运动技能可通过深度强化学习（DRL）由神经网络作为状态-动作映射习得。尽管状态观测量的选择至关重要，但目前尚缺乏定量分析。本文提出一种系统性显著性分析方法，定量评估DRL习得运动技能中不同反馈状态的相对重要性。该方法能够识别平衡恢复、慢跑、跳跃、踱步和奔跑步态等运动技能中最关键的反馈状态。仅使用关节位置、重力向量、基座线速度与角速度等关键状态，仿真四足机器人即可在多种测试场景下实现不同技能的鲁棒表现。基于任务性能指标的基准测试表明：使用关键状态学习的运动技能可达到与全状态学习相当的性能水平；若关键状态缺失，任务性能或学习成功率将显著下降。本研究为状态观测量与特定运动技能类型之间的关系提供了定量洞见，可作为机器人运动学习的指导准则。所提方法适用于任意可微状态-动作映射（如基于神经网络的控制策略），能以最小传感依赖性实现广泛运动技能的学习。