Learning an effective policy to control high-dimensional, overactuated systems is a significant challenge for deep reinforcement learning algorithms. Such control scenarios are often observed in the neural control of vertebrate musculoskeletal systems. The study of these control mechanisms will provide insights into the control of high-dimensional, overactuated systems. The coordination of actuators, known as muscle synergies in neuromechanics, is considered a presumptive mechanism that simplifies the generation of motor commands. The dynamical structure of a system is the basis of its function, allowing us to derive a synergistic representation of actuators. Motivated by this theory, we propose the Dynamical Synergistic Representation (DynSyn) algorithm. DynSyn aims to generate synergistic representations from dynamical structures and perform task-specific, state-dependent adaptation to the representations to improve motor control. We demonstrate DynSyn's efficiency across various tasks involving different musculoskeletal models, achieving state-of-the-art sample efficiency and robustness compared to baseline algorithms. DynSyn generates interpretable synergistic representations that capture the essential features of dynamical structures and demonstrates generalizability across diverse motor tasks.

翻译：学习一种有效策略来控制高维过驱动系统，是深度强化学习算法面临的一项重大挑战。此类控制场景常见于脊椎动物肌肉骨骼系统的神经控制中。研究这些控制机制将为高维过驱动系统的控制提供重要见解。执行器之间的协调（在神经力学中称为肌肉协同）被认为是一种简化运动指令生成的推定机制。系统的动力学结构是其功能的基础，使我们能够推导出执行器的协同表征。受此理论启发，我们提出了动态协同表征（DynSyn）算法。DynSyn旨在从动力学结构中生成协同表征，并对这些表征进行任务特定、状态依赖的适应性调整，以改进运动控制。我们在涉及不同肌肉骨骼模型的各种任务中验证了DynSyn的高效性，相较于基线算法，实现了最先进的样本效率和鲁棒性。DynSyn生成的协同表征具有可解释性，能够捕捉动力学结构的基本特征，并在多样化的运动任务中展现出良好的泛化能力。