Spiking Neural Networks (SNNs) are recognized as the candidate for the next-generation neural networks due to their bio-plausibility and energy efficiency. Recently, researchers have demonstrated that SNNs are able to achieve nearly state-of-the-art performance in image recognition tasks using surrogate gradient training. However, some essential questions exist pertaining to SNNs that are little studied: Do SNNs trained with surrogate gradient learn different representations from traditional Artificial Neural Networks (ANNs)? Does the time dimension in SNNs provide unique representation power? In this paper, we aim to answer these questions by conducting a representation similarity analysis between SNNs and ANNs using Centered Kernel Alignment (CKA). We start by analyzing the spatial dimension of the networks, including both the width and the depth. Furthermore, our analysis of residual connections shows that SNNs learn a periodic pattern, which rectifies the representations in SNNs to be ANN-like. We additionally investigate the effect of the time dimension on SNN representation, finding that deeper layers encourage more dynamics along the time dimension. We also investigate the impact of input data such as event-stream data and adversarial attacks. Our work uncovers a host of new findings of representations in SNNs. We hope this work will inspire future research to fully comprehend the representation power of SNNs. Code is released at https://github.com/Intelligent-Computing-Lab-Yale/SNNCKA.

翻译：脉冲神经网络（SNN）因其生物合理性和能效性，被认为是下一代神经网络的候选模型。近年来，研究者已证明SNN通过替代梯度训练能在图像识别任务中达到接近最先进的性能。然而，关于SNN仍存在一些鲜有研究的关键问题：使用替代梯度训练的SNN是否与传统人工神经网络（ANN）学习到不同的表征？SNN中的时间维度是否具有独特的表征能力？本文通过使用居中核对齐（CKA）方法对SNN与ANN进行表征相似性分析，旨在回答这些问题。我们首先从网络的空间维度（包括宽度和深度）展开分析。此外，对残差连接的分析显示，SNN学习到一种周期性模式，这使其表征趋于类似ANN。我们还研究了时间维度对SNN表征的影响，发现深层网络沿着时间维度表现出更强的动态性。同时，我们考察了事件流数据、对抗攻击等输入数据的影响。本工作揭示了SNN表征的一系列新发现，期望能启发未来研究深入理解SNN的表征能力。代码已发布在 https://github.com/Intelligent-Computing-Lab-Yale/SNNCKA。