Event-based cameras are attracting significant interest as they provide rich edge information, high dynamic range, and high temporal resolution. Many state-of-the-art event-based algorithms rely on splitting the events into fixed groups, resulting in the omission of crucial temporal information, particularly when dealing with diverse motion scenarios (\eg, high/low speed).In this work, we propose SpikeSlicer, a novel-designed plug-and-play event processing method capable of splitting events stream adaptively.SpikeSlicer utilizes a low-energy spiking neural network (SNN) to trigger event slicing. To guide the SNN to fire spikes at optimal time steps, we propose the Spiking Position-aware Loss (SPA-Loss) to modulate the neuron's state. Additionally, we develop a Feedback-Update training strategy that refines the slicing decisions using feedback from the downstream artificial neural network (ANN). Extensive experiments demonstrate that our method yields significant performance improvements in event-based object tracking and recognition. Notably, SpikeSlicer provides a brand-new SNN-ANN cooperation paradigm, where the SNN acts as an efficient, low-energy data processor to assist the ANN in improving downstream performance, injecting new perspectives and potential avenues of exploration.
翻译:基于事件的相机因其提供丰富的边缘信息、高动态范围和高时间分辨率而备受关注。许多先进的事件驱动算法依赖于将事件分割为固定组,导致关键时间信息的丢失,尤其是在处理多样化运动场景(例如高速/低速)时。本文提出SpikeSlicer,一种新颖设计的即插即用事件处理方法,能够自适应地分割事件流。SpikeSlicer利用低能耗的脉冲神经网络(SNN)触发事件切片。为引导SNN在最优时间步发放脉冲,我们提出脉冲位置感知损失(SPA-Loss)来调制神经元状态。此外,我们开发了一种反馈更新训练策略,利用下游人工神经网络(ANN)的反馈来优化切片决策。大量实验表明,我们的方法在基于事件的目标跟踪与识别任务中带来了显著的性能提升。值得注意的是,SpikeSlicer提供了一种全新的SNN-ANN协作范式,其中SNN作为高效、低能耗的数据处理器,协助ANN提升下游性能,注入了新的研究视角与探索潜力。