Leveraging the low-power, event-driven computation and the inherent temporal dynamics, spiking neural networks (SNNs) are potentially ideal solutions for processing dynamic and asynchronous signals from event-based sensors. However, due to the challenges in training and the restrictions in architectural design, there are limited examples of competitive SNNs in the realm of event-based dense prediction when compared to artificial neural networks (ANNs). In this paper, we present an efficient spiking encoder-decoder network designed for large-scale event-based semantic segmentation tasks. This is achieved by optimizing the encoder using a hierarchical search method. To enhance learning from dynamic event streams, we harness the inherent adaptive threshold of spiking neurons to modulate network activation. Moreover, we introduce a dual-path Spiking Spatially-Adaptive Modulation (SSAM) block, specifically designed to enhance the representation of sparse events, thereby considerably improving network performance. Our proposed network achieves a 72.57% mean intersection over union (MIoU) on the DDD17 dataset and a 57.22% MIoU on the recently introduced, larger DSEC-Semantic dataset. This performance surpasses the current state-of-the-art ANNs by 4%, whilst consuming significantly less computational resources. To the best of our knowledge, this is the first study demonstrating SNNs outperforming ANNs in demanding event-based semantic segmentation tasks, thereby establishing the vast potential of SNNs in the field of event-based vision. Our source code will be made publicly accessible.

翻译：利用低功耗、事件驱动的计算特性及其固有的时间动态性，脉冲神经网络(SNNs)有望成为处理事件传感器产生的动态异步信号的理想方案。然而，由于训练困难及架构设计的局限性，相较于人工神经网络(ANNs)，在事件驱动密集预测领域鲜有具备竞争力的SNN模型。本文提出了一种面向大规模事件驱动语义分割任务的高效脉冲编码器-解码器网络，通过层级搜索方法优化编码器实现高效性能。为增强对动态事件流的学习能力，我们利用脉冲神经元内在的自适应阈值来调控网络激活。此外，创新性地引入双路径脉冲空间自适应调制(SSAM)模块，该模块专为增强稀疏事件表征设计，显著提升了网络性能。所提网络在DDD17数据集上达到72.57%的平均交并比(MIoU)，在最新发布的大型DSEC-Semantic数据集上达到57.22%的MIoU，超越当前最先进的ANNs网络4%的同时大幅降低计算资源消耗。据我们所知，这是首个证明SNN在复杂事件驱动语义分割任务中性能超越ANNs的研究，充分展现了SNN在事件驱动视觉领域的巨大潜力。我们的源代码将公开提供。