This article studies the expressive power of spiking neural networks where information is encoded in the firing time of neurons. The implementation of spiking neural networks on neuromorphic hardware presents a promising choice for future energy-efficient AI applications. However, there exist very few results that compare the computational power of spiking neurons to arbitrary threshold circuits and sigmoidal neurons. Additionally, it has also been shown that a network of spiking neurons is capable of approximating any continuous function. By using the Spike Response Model as a mathematical model of a spiking neuron and assuming a linear response function, we prove that the mapping generated by a network of spiking neurons is continuous piecewise linear. We also show that a spiking neural network can emulate the output of any multi-layer (ReLU) neural network. Furthermore, we show that the maximum number of linear regions generated by a spiking neuron scales exponentially with respect to the input dimension, a characteristic that distinguishes it significantly from an artificial (ReLU) neuron. Our results further extend the understanding of the approximation properties of spiking neural networks and open up new avenues where spiking neural networks can be deployed instead of artificial neural networks without any performance loss.

翻译：本文研究了脉冲神经网络的表达能力，其中信息以神经元的脉冲发射时间进行编码。脉冲神经网络在神经形态硬件上的实现为未来节能型人工智能应用提供了有前景的选择。然而，目前鲜有研究将脉冲神经元的计算能力与任意阈值电路及Sigmoid型神经元进行比较。此外，已有研究表明脉冲神经元网络能够逼近任意连续函数。通过采用脉冲响应模型作为脉冲神经元的数学模型，并假设线性响应函数，我们证明了脉冲神经元网络生成的映射是连续分段线性函数。我们还表明，脉冲神经网络可以模拟任何多层（ReLU）神经网络的输出。进一步地，我们揭示了单个脉冲神经元生成的线性区域最大数量随输入维度呈指数级增长，这一特性使其与人工（ReLU）神经元显著区分。我们的结果拓展了对脉冲神经网络逼近性质的理解，并开辟了在性能无损前提下用脉冲神经网络替代人工神经网络的新途径。