Spiking Neural Networks (SNNs) have been proposed as biologically plausible and energy-efficient alternatives to conventional Artificial Neural Networks (ANNs). However, the training of SNN usually relies on surrogate gradients due to the non-differentiability of the spike function, introducing approximation errors that accumulate across layers. To address this challenge, we extend the work on convexification of parallel feedforward threshold networks to parallel recurrent threshold networks, which subsume parallel SNNs as a structured special case. Building on this theoretical framework, we propose a parameter reconstruction algorithm for SNN training that demonstrates consistent and significant advantages across various tasks, both as a standalone method and in combination with surrogate-gradient training. The ablations further demonstrate the data scalability and robustness to model configurations of our training algorithm, pointing toward its potential in large-scale SNN training.
翻译:脉冲神经网络(SNNs)因其生物合理性和能效优势被视为传统人工神经网络(ANNs)的替代方案。然而,由于脉冲函数的不可微性,SNN的训练通常依赖于替代梯度,这会在层间累积近似误差。为解决这一难题,我们将并行前馈阈值网络的凸化方法扩展至并行循环阈值网络,后者将并行SNN作为结构化特例纳入框架。基于该理论体系,我们提出了一种用于SNN训练的参数量构算法,该算法在多种任务中作为独立方法及与替代梯度训练相结合时均展现出显著且一致的优势。消融实验进一步验证了本训练算法对数据规模的可扩展性及对模型配置的鲁棒性,表明其在大规模SNN训练中的潜力。