Spiking Neural Networks (SNNs) have garnered increasing attention as one of bio-inspired models due to their great potential in neuromorphic computing and sparse computation. Many practical algorithms and techniques have been developed; however, theoretical understandings of the generalization, that is, the extent to which SNNs perform well on unseen data, are far from clear. Recent advances disclosed an excitation-dependent and architecture-related generalization bound such that the Rademacher complexity of SNNs with stochastic firing can be upper bounded by an exponential function relative to the excitation probability and the architecture depth. In this paper, we theoretically investigate the generalization bounds of SNNs with several integration-and-fire schemes via Rademacher complexity. We recognize that the empirical Rademacher complexity of SNNs is close to the SNN configurations, which is exponential to the network depth and the maximum time duration of received spike sequences, superlinear and subquadratic to the network width, polynomial to the parameter norm, inverse-linear to the number of training samples, and independent of the computations within spiking neurons, achieving a more precise rate than conventional studies. Our theoretical results may support the scope of SNN theories and shed some insight into the development of SNNs.

翻译：脉冲神经网络（SNNs）作为一类受生物启发的模型，因其在神经形态计算和稀疏计算中的巨大潜力而日益受到关注。尽管已经开发了许多实用算法和技术，但关于其泛化能力（即SNNs在未见数据上表现良好的程度）的理论理解仍远未清晰。近年来的进展揭示了一个与激发概率和架构深度相关的指数函数上界，即具有随机放电机制的SNNs的Rademacher复杂度可以被该指数函数所约束的激发依赖性和架构相关泛化界。本文通过Rademacher复杂度从理论上研究了采用多种整合-放电方案的SNNs的泛化界。我们认识到，SNNs的经验Rademacher复杂度接近其网络配置，且与网络深度和接收脉冲序列的最大持续时间呈指数关系，与网络宽度呈超线性且次二次关系，与参数范数呈多项式关系，与训练样本数呈反线性关系，并且与脉冲神经元内部计算无关，从而获得了比传统研究更精确的速率。我们的理论结果可能支撑SNN理论的范围，并为SNNs的发展提供一些见解。