Developing autonomous agents that can interact with changing environments is an open challenge in machine learning. Robustness is particularly important in these settings as agents are often fit offline on expert demonstrations but deployed online where they must generalize to the closed feedback loop within the environment. In this work, we explore the application of recurrent neural networks to tasks of this nature and understand how a parameterization of their recurrent connectivity influences robustness in closed-loop settings. Specifically, we represent the recurrent connectivity as a function of rank and sparsity and show both theoretically and empirically that modulating these two variables has desirable effects on network dynamics. The proposed low-rank, sparse connectivity induces an interpretable prior on the network that proves to be most amenable for a class of models known as closed-form continuous-time neural networks (CfCs). We find that CfCs with fewer parameters can outperform their full-rank, fully-connected counterparts in the online setting under distribution shift. This yields memory-efficient and robust agents while opening a new perspective on how we can modulate network dynamics through connectivity.

翻译：开发能够与变化环境交互的自主智能体是机器学习领域的一项开放挑战。鲁棒性在此类场景中尤为重要，因为智能体通常基于专家演示进行离线训练，却需在在线环境中部署，并必须泛化至环境中的闭环反馈回路。本研究探索了循环神经网络在此类任务中的应用，并揭示了其循环连接的参数化方式如何影响闭环场景下的鲁棒性。具体而言，我们将循环连接表示为秩与稀疏度的函数，并从理论与实证角度证明：调节这两个变量对网络动力学具有理想效果。所提出的低秩稀疏连接为网络引入了可解释先验，该先验对一类名为闭式连续时间神经网络（CfCs）的模型尤为适用。研究发现，在分布偏移的在线场景下，参数更少的CfCs能够超越其满秩全连接对应模型。这既产生了内存高效且鲁棒的智能体，也为通过连接调控网络动力学提供了全新视角。