Designing controllers to achieve natural motor capabilities for multi-joint robots is a significant challenge. However, animals in nature are naturally with basic motor abilities and can master various complex motor skills through acquired learning. On the basis of analyzing the mechanism of the central motor system in mammals, we propose a novel pre-training reinforcement learning algorithm that enables robots to learn rich motor skills and apply them to complex task environments without relying on external data. We first design a skill based network similar to the cerebellum by utilizing the selection mechanism of voluntary movements in the basal ganglia and the basic motor regulation ability of the cerebellum. Subsequently, by imitating the structure of advanced centers in the central motor system, we propose a high-level policy to generate different skill combinations, thereby enabling the robot to acquire natural motor abilities. We conduct experiments on 4 types of robots and 22 task environments, and the results show that the proposed method can enable different types of robots to achieve flexible motor skills. Overall, our research provides a promising framework for the design of neural network motor controllers.

翻译：设计控制器以实现多关节机器人的自然运动能力是一项重大挑战。然而，自然界中的动物天生具备基础运动能力，并能通过后天学习掌握多种复杂运动技能。在分析哺乳动物中枢运动系统机制的基础上，我们提出了一种新型预训练强化学习算法，使机器人无需依赖外部数据即可学习丰富的运动技能并将其应用于复杂任务环境。我们首先利用基底神经节对随意运动的选择机制和小脑的基本运动调节能力，设计了类似小脑的技能基网络。随后，通过模仿中枢运动系统中高级中枢的结构，我们提出了一种高层策略以生成不同的技能组合，从而使机器人获得自然运动能力。我们在4种机器人及22个任务环境中进行了实验，结果表明所提方法能使不同类型机器人实现灵活的运动技能。总体而言，本研究为神经网络运动控制器的设计提供了一种有前景的框架。