Spiking Neural Networks (SNNs) have the potential for rich spatio-temporal signal processing thanks to exploiting both spatial and temporal parameters. The temporal dynamics such as time constants of the synapses and neurons and delays have been recently shown to have computational benefits that help reduce the overall number of parameters required in the network and increase the accuracy of the SNNs in solving temporal tasks. Optimizing such temporal parameters, for example, through gradient descent, gives rise to a temporal architecture for different problems. As has been shown in machine learning, to reduce the cost of optimization, architectural biases can be applied, in this case in the temporal domain. Such inductive biases in temporal parameters have been found in neuroscience studies, highlighting a hierarchy of temporal structure and input representation in different layers of the cortex. Motivated by this, we propose to impose a hierarchy of temporal representation in the hidden layers of SNNs, highlighting that such an inductive bias improves their performance. We demonstrate the positive effects of temporal hierarchy in the time constants of feed-forward SNNs applied to temporal tasks (Multi-Time-Scale XOR and Keyword Spotting, with a benefit of up to 4.1% in classification accuracy). Moreover, we show that such architectural biases, i.e. hierarchy of time constants, naturally emerge when optimizing the time constants through gradient descent, initialized as homogeneous values. We further pursue this proposal in temporal convolutional SNNs, by introducing the hierarchical bias in the size and dilation of temporal kernels, giving rise to competitive results in popular temporal spike-based datasets.

翻译：脉冲神经网络（SNNs）通过同时利用空间和时间参数，具备丰富的时空信号处理潜力。突触和神经元的时间常数、延迟等时间动态特性近期被证明具有计算优势，有助于减少网络所需的总参数量，并提升SNNs在解决时序任务中的准确性。通过梯度下降等方法优化此类时间参数，可为不同问题构建时间架构。正如机器学习领域所示，为降低优化成本，可以引入架构偏置，此处即在时间域中施加。神经科学研究已发现时间参数中的此类归纳偏置，揭示了大脑皮层不同层次中时间结构与输入表征的层级性。受此启发，我们提出在SNNs的隐藏层中施加时间表征的层次结构，并证明这种归纳偏置能提升其性能。我们在应用于时序任务（多时间尺度异或运算和关键词检测）的前馈SNNs中，展示了时间常数层次结构带来的积极效果（分类准确率最高提升4.1%）。此外，我们发现当以均匀值初始化并通过梯度下降优化时间常数时，此类架构偏置（即时间常数层次结构）会自然涌现。我们进一步在时序卷积SNNs中推进该方案，通过在时间核的尺寸和扩张率中引入层次化偏置，在主流脉冲时序数据集上取得了具有竞争力的结果。