In recent years, spiking neural networks (SNNs) have gained momentum due to their high potential in time-series processing combined with minimal energy consumption. However, they still lack a dedicated and efficient training algorithm. The popular backpropagation with surrogate gradients, adapted from stochastic gradient descent (SGD)-derived algorithms, has several drawbacks when used as an optimizer for SNNs. Specifically, the approximation introduced by the use of surrogate gradients leads to numerical imprecision, poor tracking of SNN firing times at training time, and, in turn, poor scalability. In this paper, we propose a novel SNN training method based on the alternating direction method of multipliers (ADMM). Our ADMM-based training aims to solve the problem of the SNN step function's non-differentiability by taking an entirely new approach with respect to gradient backpropagation. For the first time, we formulate the SNN training problem as an ADMM-based iterative optimization, derive closed-form updates, and empirically show the optimizer's convergence, its great potential, and discuss future and promising research directions to improve the method to different layer types and deeper architectures.

翻译：近年来，脉冲神经网络（SNNs）因其在时序处理方面的巨大潜力及极低的能耗而备受关注。然而，该领域仍缺乏专门且高效的训练算法。目前广泛采用的基于替代梯度的反向传播方法，源自随机梯度下降（SGD）类算法，在作为SNN优化器时存在若干缺陷。具体而言，替代梯度引入的近似会导致数值不精确、训练时对SNN脉冲发放时刻的追踪能力不足，进而影响算法的可扩展性。本文提出一种基于交替方向乘子法（ADMM）的新型SNN训练方法。该训练方法通过采用完全不同于梯度反向传播的新思路，旨在解决SNN阶跃函数的不可微问题。我们首次将SNN训练问题构建为基于ADMM的迭代优化形式，推导出闭式更新规则，并通过实验验证了优化器的收敛性及其巨大潜力。最后，本文探讨了未来有望改进该方法的研究方向，包括将其拓展至不同层类型及更深层网络架构的可行性。