As one of the energy-efficient alternatives of conventional neural networks (CNNs), spiking neural networks (SNNs) have gained more and more interest recently. To train the deep models, some effective batch normalization (BN) techniques are proposed in SNNs. All these BNs are suggested to be used after the convolution layer as usually doing in CNNs. However, the spiking neuron is much more complex with the spatio-temporal dynamics. The regulated data flow after the BN layer will be disturbed again by the membrane potential updating operation before the firing function, i.e., the nonlinear activation. Therefore, we advocate adding another BN layer before the firing function to normalize the membrane potential again, called MPBN. To eliminate the induced time cost of MPBN, we also propose a training-inference-decoupled re-parameterization technique to fold the trained MPBN into the firing threshold. With the re-parameterization technique, the MPBN will not introduce any extra time burden in the inference. Furthermore, the MPBN can also adopt the element-wised form, while these BNs after the convolution layer can only use the channel-wised form. Experimental results show that the proposed MPBN performs well on both popular non-spiking static and neuromorphic datasets. Our code is open-sourced at \href{https://github.com/yfguo91/MPBN}{MPBN}.

翻译：作为传统神经网络（CNN）的节能替代方案之一，脉冲神经网络（SNN）近年来受到越来越多的关注。为训练深层模型，研究者已在SNN中提出若干有效的批归一化（BN）技术。这些BN方法通常遵循CNN中的常见做法，被建议置于卷积层之后。然而，脉冲神经元具有更复杂的时空动态特性，BN层调控后的数据流在进入点火函数（即非线性激活）前，会再次受到膜电位更新操作的干扰。因此，我们主张在点火函数前增设一个BN层以再次归一化膜电位，称为MPBN。为消除MPBN引入的额外时间开销，我们提出一种训练-推理解耦的重参数化技术，将训练后的MPBN折叠至点火阈值中。采用该重参数化技术后，MPBN在推理阶段不会引入任何额外时间负担。此外，MPBN还可采用逐元素形式，而卷积层后的BN仅能使用通道形式。实验结果表明，所提出的MPBN在流行的非脉冲静态数据集和神经形态数据集上均表现优异。我们的代码已开源至 \href{https://github.com/yfguo91/MPBN}{MPBN}。