Learning an effective policy to control high-dimensional, overactuated systems is a significant challenge for deep reinforcement learning algorithms. Such control scenarios are often observed in the neural control of vertebrate musculoskeletal systems. The study of these control mechanisms will provide insights into the control of high-dimensional, overactuated systems. The coordination of actuators, known as muscle synergies in neuromechanics, is considered a presumptive mechanism that simplifies the generation of motor commands. The dynamical structure of a system is the basis of its function, allowing us to derive a synergistic representation of actuators. Motivated by this theory, we propose the Dynamical Synergistic Representation (DynSyn) algorithm. DynSyn aims to generate synergistic representations from dynamical structures and perform task-specific, state-dependent adaptation to the representations to improve motor control. We demonstrate DynSyn's efficiency across various tasks involving different musculoskeletal models, achieving state-of-the-art sample efficiency and robustness compared to baseline algorithms. DynSyn generates interpretable synergistic representations that capture the essential features of dynamical structures and demonstrates generalizability across diverse motor tasks.

翻译：学习一种有效策略来控制高维过驱动系统是深度强化学习算法面临的重大挑战。这类控制场景常见于脊椎动物肌肉骨骼系统的神经控制研究中。对这些控制机制的研究将为高维过驱动系统的控制提供重要洞见。执行器间的协调机制（在神经力学中称为肌肉协同）被认为是一种简化运动指令生成的推定机制。系统的动力学结构是其功能实现的基础，使我们能够推导出执行器的协同表征。受该理论启发，我们提出动态协同表征（DynSyn）算法。DynSyn旨在从动力学结构中生成协同表征，并对表征进行任务特异性、状态依赖性的自适应调整以提升运动控制性能。我们在涉及不同肌肉骨骼模型的多项任务中验证了DynSyn的效能，相较于基线算法实现了最优的样本效率与鲁棒性。DynSyn生成的协同表征具有可解释性，能够捕捉动力学结构的本质特征，并在多样化运动任务中展现出良好的泛化能力。