In recent years, newly developed methods to train spiking neural networks (SNNs) have rendered them as a plausible alternative to Artificial Neural Networks (ANNs) in terms of accuracy, while at the same time being much more energy efficient at inference and potentially at training time. However, it is still unclear what constitutes a good initialisation for an SNN. We often use initialisation schemes developed for ANN training which are often inadequate and require manual tuning. In this paper, we attempt to tackle this issue by using techniques from the ANN initialisation literature as well as computational neuroscience results. We show that the problem of weight initialisation for ANNs is a more nuanced problem than it is for ANNs due to the spike-and-reset non-linearity of SNNs and the firing rate collapse problem. We firstly identify and propose several solutions to the firing rate collapse problem under different sets of assumptions which successfully solve the issue by leveraging classical random walk and Wiener processes results. Secondly, we devise a general strategy for SNN initialisation which combines variance propagation techniques from ANNs and different methods to obtain the expected firing rate and membrane potential distribution based on diffusion and shot-noise approximations. Altogether, we obtain theoretical results to solve the SNN initialisation which consider the membrane potential distribution in the presence of a threshold. Yet, to what extent can these methods be successfully applied to SNNs on real datasets remains an open question.

翻译：近年来，新开发的脉冲神经网络训练方法使其在精度上成为人工神经网络（ANNs）的可行替代方案，同时在推理及潜在训练过程中具备更高的能效。然而，脉冲神经网络（SNNs）的合理初始化方案仍不明确。我们常采用为ANNs设计的初始化方法，但这些方法往往不适用且需要手动调整。本文尝试通过借鉴ANNs初始化文献中的技术以及计算神经科学的研究成果来解决这一问题。研究表明，由于SNNs的脉冲-重置非线性特性及发放率崩溃现象，其权重初始化问题比ANNs更为复杂。首先，我们识别了不同假设条件下的发放率崩溃问题，并基于经典随机游走与维纳过程结果提出多种解决方案，成功消除该问题。其次，我们设计了一种通用的SNN初始化策略，该策略融合了ANNs的方差传播技术，以及基于扩散和散粒噪声近似获得期望发放率和膜电位分布的不同方法。总体而言，我们获得了考虑阈值条件下膜电位分布的SNN初始化理论结果。然而，这些方法在真实数据集上能否成功应用于SNNs仍是一个开放性问题。