The rise of mobility, IoT and wearables has shifted processing to the edge of the sensors, driven by the need to reduce latency, communication costs and overall energy consumption. While deep learning models have achieved remarkable results in various domains, their deployment at the edge for real-time applications remains computationally expensive. Neuromorphic computing emerges as a promising paradigm shift, characterized by co-localized memory and computing as well as event-driven asynchronous sensing and processing. In this work, we demonstrate the possibility of solving the ubiquitous computer vision task of object detection at the edge with low-power requirements, using the event-based N-Caltech101 dataset. We present the first instance of an on-chip spiking neural network for event-based face detection deployed on the SynSense Speck neuromorphic chip, which comprises both an event-based sensor and a spike-based asynchronous processor implementing Integrate-and-Fire neurons. We show how to reduce precision discrepancies between off-chip clock-driven simulation used for training and on-chip event-driven inference. This involves using a multi-spike version of the Integrate-and-Fire neuron on simulation, where spikes carry values that are proportional to the extent the membrane potential exceeds the firing threshold. We propose a robust strategy to train spiking neural networks with back-propagation through time using multi-spike activation and firing rate regularization and demonstrate how to decode output spikes into bounding boxes. We show that the power consumption of the chip is directly proportional to the number of synaptic operations in the spiking neural network, and we explore the trade-off between power consumption and detection precision with different firing rate regularization, achieving an on-chip face detection mAP[0.5] of ~0.6 while consuming only ~20 mW.

翻译：移动性、物联网和可穿戴设备的兴起将处理任务推向传感器边缘，这得益于对降低延迟、通信成本和总体能耗的需求。尽管深度学习模型在各领域取得了显著成果，但其在边缘设备上用于实时应用时仍面临计算成本高的挑战。神经形态计算作为一种有前景的范式转变，其特点包括内存与计算共定位以及事件驱动的异步感知与处理。在本研究中，我们利用基于事件的N-Caltech101数据集，展示了解决边缘设备上通用计算机视觉任务（即目标检测）且满足低功耗要求的可能性。我们首次在SynSense Speck神经形态芯片上部署了用于基于事件人脸检测的片上脉冲神经网络，该芯片集成了基于事件的传感器和基于脉冲的异步处理器，实现了积分-放电神经元。我们展示了如何减少用于训练的片外时钟驱动仿真与片上事件驱动推理之间的精度差异。这涉及在仿真中使用多脉冲版本的积分-放电神经元，其中脉冲携带的值与膜电位超过放电阈值的程度成正比。我们提出了一种稳健的策略，通过时间反向传播训练脉冲神经网络，采用多脉冲激活和放电频率正则化，并演示如何将输出脉冲解码为边界框。结果表明，芯片的功耗与脉冲神经网络中的突触操作数量成正比，我们探讨了不同放电频率正则化下功耗与检测精度之间的权衡，在仅消耗约20 mW功耗的情况下，实现了片上人脸检测mAP[0.5]约0.6的性能。