The highly sparse activations in Spiking Neural Networks (SNNs) can provide tremendous energy efficiency benefits when carefully exploited in hardware. The behavior of sparsity in SNNs is uniquely shaped by the dataset and training hyperparameters. This work reveals novel insights into the impacts of training on hardware performance. Specifically, we explore the trade-offs between model accuracy and hardware efficiency. We focus on three key hyperparameters: surrogate gradient functions, beta, and membrane threshold. Results on an FPGA-based hardware platform show that the fast sigmoid surrogate function yields a lower firing rate with similar accuracy compared to the arctangent surrogate on the SVHN dataset. Furthermore, by cross-sweeping the beta and membrane threshold hyperparameters, we can achieve a 48% reduction in hardware-based inference latency with only 2.88% trade-off in inference accuracy compared to the default setting. Overall, this study highlights the importance of fine-tuning model hyperparameters as crucial for designing efficient SNN hardware accelerators, evidenced by the fine-tuned model achieving a 1.72x improvement in accelerator efficiency (FPS/W) compared to the most recent work.

翻译：脉冲神经网络（SNNs）中高度稀疏的激活特性，在硬件中精心利用时可带来巨大的能效优势。SNN中稀疏性的行为由数据集和训练超参数独特塑造。本研究揭示了训练对硬件性能影响的新颖见解，具体探讨了模型精度与硬件效率之间的权衡。我们聚焦于三个关键超参数：替代梯度函数、beta值和膜阈值。基于FPGA硬件平台的实验结果表明，在SVHN数据集上，快速sigmoid替代函数相比反正切替代函数能实现更低的发放率且精度相近。此外，通过交叉扫描beta值和膜阈值超参数，与默认设置相比，我们可实现硬件推理延迟降低48%，而推理精度仅牺牲2.88%。总体而言，本研究强调了微调模型超参数对设计高效SNN硬件加速器至关重要，经微调的模型与最新工作相比，加速器效率（FPS/W）提升了1.72倍。