Recent years have emerged a surge of interest in SNNs owing to their remarkable potential to handle time-dependent and event-driven data. The performance of SNNs hinges not only on selecting an apposite architecture and fine-tuning connection weights, similar to conventional ANNs, but also on the meticulous configuration of intrinsic structures within spiking computations. However, there has been a dearth of comprehensive studies examining the impact of intrinsic structures. Consequently, developers often find it challenging to apply a standardized configuration of SNNs across diverse datasets or tasks. This work delves deep into the intrinsic structures of SNNs. Initially, we unveil two pivotal components of intrinsic structures: the integration operation and firing-reset mechanism, by elucidating their influence on the expressivity of SNNs. Furthermore, we draw two key conclusions: the membrane time hyper-parameter is intimately linked to the eigenvalues of the integration operation, dictating the functional topology of spiking dynamics, and various hyper-parameters of the firing-reset mechanism govern the overall firing capacity of an SNN, mitigating the injection ratio or sampling density of input data. These findings elucidate why the efficacy of SNNs hinges heavily on the configuration of intrinsic structures and lead to a recommendation that enhancing the adaptability of these structures contributes to improving the overall performance and applicability of SNNs. Inspired by this recognition, we propose two feasible approaches to enhance SNN learning. These involve leveraging self-connection architectures and employing stochastic spiking neurons to augment the adaptability of the integration operation and firing-reset mechanism, respectively. We verify the effectiveness of the proposed methods from perspectives of theory and practice.
翻译:近年来,由于处理时间依赖性和事件驱动数据的显著潜力,脉冲神经网络引起了广泛关注。与人工神经网络类似,SNN的性能不仅依赖于选择合适的架构和微调连接权重,还取决于脉冲计算中内在结构的精细配置。然而,目前缺乏系统研究考察内在结构的影响,导致开发者难以在不同数据集或任务中应用标准化的SNN配置。本文深入探究了SNN的内在结构。首先,我们揭示了内在结构的两个关键组成部分:积分操作和发放-重置机制,阐明了它们对SNN表达力的影响。此外,我们得出两个关键结论:膜时间超参数与积分操作的特征值密切相关,决定了脉冲动力学的功能拓扑;发放-重置机制的不同超参数则控制着SNN的整体发放容量,调节输入数据的注入比率或采样密度。这些发现解释了SNN效能为何高度依赖于内在结构的配置,并提出增强这些结构的适应性有助于提升SNN整体性能与适用性的建议。受此启发,我们提出了两种可行方法以增强SNN学习:利用自连接架构提升积分操作的适应性,以及采用随机发放神经元增强发放-重置机制的适应性。我们从理论和实践角度验证了所提方法的有效性。