In neural circuits, recurrent connectivity plays a crucial role in network function and stability. However, existing recurrent spiking neural networks (RSNNs) are often constructed by random connections without optimization. While RSNNs can produce rich dynamics that are critical for memory formation and learning, systemic architectural optimization of RSNNs is still an open challenge. We aim to enable systematic design of large RSNNs via a new scalable RSNN architecture and automated architectural optimization. We compose RSNNs based on a layer architecture called Sparsely-Connected Recurrent Motif Layer (SC-ML) that consists of multiple small recurrent motifs wired together by sparse lateral connections. The small size of the motifs and sparse inter-motif connectivity leads to an RSNN architecture scalable to large network sizes. We further propose a method called Hybrid Risk-Mitigating Architectural Search (HRMAS) to systematically optimize the topology of the proposed recurrent motifs and SC-ML layer architecture. HRMAS is an alternating two-step optimization process by which we mitigate the risk of network instability and performance degradation caused by architectural change by introducing a novel biologically-inspired "self-repairing" mechanism through intrinsic plasticity. The intrinsic plasticity is introduced to the second step of each HRMAS iteration and acts as unsupervised fast self-adaptation to structural and synaptic weight modifications introduced by the first step during the RSNN architectural "evolution". To the best of the authors' knowledge, this is the first work that performs systematic architectural optimization of RSNNs. Using one speech and three neuromorphic datasets, we demonstrate the significant performance improvement brought by the proposed automated architecture optimization over existing manually-designed RSNNs.

翻译：在神经回路中，递归连接对网络功能与稳定性起着关键作用。然而，现有递归脉冲神经网络（RSNN）通常通过随机连接构建，缺乏优化机制。尽管RSNN能产生对记忆形成与学习至关重要的丰富动力学特性，但其系统性架构优化仍是一个开放挑战。本文旨在通过新型可扩展RSNN架构与自动化架构优化，实现大规模RSNN的系统化设计。我们基于一种称为稀疏连接递归基序层（SC-ML）的层级架构构建RSNN，该架构由多个小型递归基序通过稀疏侧向连接相互耦合而成。基序的小型化设计与基序间稀疏连接特性使该RSNN架构具备可扩展至大规模网络的能力。我们进一步提出混合风险缓解架构搜索（HRMAS）方法，对递归基序与SC-ML层级架构的拓扑结构进行系统性优化。HRMAS是一种交替式两步优化过程：通过引入受生物启发的"自修复"机制（基于内在可塑性），缓解架构变更导致的网络不稳定与性能退化风险。在每次HRMAS迭代的第二步中引入内在可塑性，使其作为无监督快速自适应机制，应对RSNN架构"进化"过程中第一步产生的结构性与突触权重修改。据作者所知，这是首个对RSNN进行系统性架构优化的研究工作。通过使用一个语音数据集与三个神经形态数据集，我们证明了所提出的自动化架构优化相较于现有手动设计RSNN所带来的显著性能提升。