Recurrent neural network-based reinforcement learning systems are capable of complex motor control tasks such as locomotion and manipulation, however, much of their underlying mechanisms still remain difficult to interpret. Our aim is to leverage computational neuroscience methodologies to understanding the population-level activity of robust robot locomotion controllers. Our investigation begins by analyzing topological structure, discovering that fragile controllers have a higher number of fixed points with unstable directions, resulting in poorer balance when instructed to stand in place. Next, we analyze the forced response of the system by applying targeted neural perturbations along directions of dominant population-level activity. We find evidence that recurrent state dynamics are structured and low-dimensional during walking, which aligns with primate studies. Additionally, when recurrent states are perturbed to zero, fragile agents continue to walk, which is indicative of a stronger reliance on sensory input and weaker recurrence.

翻译：基于递归神经网络的强化学习系统能够执行复杂的运动控制任务（如 locomotion 与 manipulation），但其内在机制仍难以解释。本研究旨在利用计算神经科学方法理解鲁棒机器人运动控制器的群体水平活动。我们首先分析拓扑结构，发现脆弱控制器具有更多含不稳定方向的固定点，导致其在原地站立指令下平衡性较差。随后，通过沿主导群体活动方向施加靶向神经扰动，分析系统的强迫响应。研究发现步行过程中递归状态动态呈现结构化及低维特性，这与灵长类动物研究结果一致。此外，当递归状态被扰动至零时，脆弱智能体仍能继续行走，表明其对感官输入的依赖更强而递归作用较弱。