Early sensory systems in the brain rapidly adapt to fluctuating input statistics, which requires recurrent communication between neurons. Mechanistically, such recurrent communication is often indirect and mediated by local interneurons. In this work, we explore the computational benefits of mediating recurrent communication via interneurons compared with direct recurrent connections. To this end, we consider two mathematically tractable recurrent linear neural networks that statistically whiten their inputs -- one with direct recurrent connections and the other with interneurons that mediate recurrent communication. By analyzing the corresponding continuous synaptic dynamics and numerically simulating the networks, we show that the network with interneurons is more robust to initialization than the network with direct recurrent connections in the sense that the convergence time for the synaptic dynamics in the network with interneurons (resp. direct recurrent connections) scales logarithmically (resp. linearly) with the spectrum of their initialization. Our results suggest that interneurons are computationally useful for rapid adaptation to changing input statistics. Interestingly, the network with interneurons is an overparameterized solution of the whitening objective for the network with direct recurrent connections, so our results can be viewed as a recurrent linear neural network analogue of the implicit acceleration phenomenon observed in overparameterized feedforward linear neural networks.

翻译：大脑早期感觉系统能够快速适应不断变化的输入统计特性，这依赖于神经元之间的递归通信。从机制上看，这种递归通信通常是非直接的，并由局部中间神经元介导。本研究探讨了通过中间神经元介导递归通信相较于直接递归连接的计算优势。为此，我们构建了两个数学上可处理的递归线性神经网络，它们能够统计白化其输入——其中一个网络采用直接递归连接，另一个网络则通过中间神经元介导递归通信。通过分析相应的连续突触动力学并数值模拟网络，我们发现：与直接递归连接的网络相比，中间神经元网络对初始化更鲁棒——中间神经元网络（或直接递归连接网络）中突触动力学的收敛时间与其初始化谱呈对数（或线性）关系。我们的结果表明，中间神经元在快速适应输入统计特性变化方面具有计算优势。有趣的是，对于直接递归连接网络的白化目标而言，中间神经元网络是其过参数化解；因此，我们的结果可视为过参数化前馈线性神经网络中隐式加速现象在递归线性神经网络中的类比。