Spiking neural networks (SNNs) have been widely used due to their strong biological interpretability and high energy efficiency. With the introduction of the backpropagation algorithm and surrogate gradient, the structure of spiking neural networks has become more complex, and the performance gap with artificial neural networks has gradually decreased. However, most SNN hardware implementations for field-programmable gate arrays (FPGAs) cannot meet arithmetic or memory efficiency requirements, which significantly restricts the development of SNNs. They do not delve into the arithmetic operations between the binary spikes and synaptic weights or assume unlimited on-chip RAM resources by using overly expensive devices on small tasks. To improve arithmetic efficiency, we analyze the neural dynamics of spiking neurons, generalize the SNN arithmetic operation to the multiplex-accumulate operation, and propose a high-performance implementation of such operation by utilizing the DSP48E2 hard block in Xilinx Ultrascale FPGAs. To improve memory efficiency, we design a memory system to enable efficient synaptic weights and membrane voltage memory access with reasonable on-chip RAM consumption. Combining the above two improvements, we propose an FPGA accelerator that can process spikes generated by the firing neuron on-the-fly (FireFly). FireFly is the first SNN accelerator that incorporates DSP optimization techniques into SNN synaptic operations. FireFly is implemented on several FPGA edge devices with limited resources but still guarantees a peak performance of 5.53TOP/s at 300MHz. As a lightweight accelerator, FireFly achieves the highest computational density efficiency compared with existing research using large FPGA devices.

翻译：脉冲神经网络因其强生物可解释性和高能效而被广泛应用。随着反向传播算法和代理梯度的引入，脉冲神经网络结构日趋复杂，与人工神经网络的性能差距逐渐缩小。然而，现有大多数基于现场可编程门阵列的SNN硬件实现在算术或存储效率方面无法满足需求，严重制约了SNN的发展。这些工作既未深入探究二进制脉冲与突触权重间的算术运算，也未考虑实际片上RAM资源限制，往往在小规模任务中使用昂贵器件假设无限资源。为提升算术效率，我们分析脉冲神经元动力学特性，将SNN算术运算泛化为乘累加运算，并利用Xilinx Ultrascale系列FPGA中的DSP48E2硬核模块提出高性能实现方案。为提升存储效率，我们设计存储系统，在合理消耗片上RAM的前提下实现高效的突触权重与膜电位访问。综合上述两项改进，我们提出可实时处理发放神经元产生脉冲的FPGA加速器FireFly。FireFly是首个将DSP优化技术融入SNN突触运算的加速器。该加速器在资源有限的边缘FPGA器件上实现，仍可在300MHz主频下保证5.53TOPS峰值性能。作为轻量级加速器，FireFly相比现有使用大型FPGA器件的研究实现了最高计算密度效率。