Event cameras are bio-inspired sensors that respond to local changes in light intensity and feature low latency, high energy efficiency, and high dynamic range. Meanwhile, Spiking Neural Networks (SNNs) have gained significant attention due to their remarkable efficiency and fault tolerance. By synergistically harnessing the energy efficiency inherent in event cameras and the spike-based processing capabilities of SNNs, their integration could enable ultra-low-power application scenarios, such as action recognition tasks. However, existing approaches often entail converting asynchronous events into conventional frames, leading to additional data mapping efforts and a loss of sparsity, contradicting the design concept of SNNs and event cameras. To address this challenge, we propose SpikePoint, a novel end-to-end point-based SNN architecture. SpikePoint excels at processing sparse event cloud data, effectively extracting both global and local features through a singular-stage structure. Leveraging the surrogate training method, SpikePoint achieves high accuracy with few parameters and maintains low power consumption, specifically employing the identity mapping feature extractor on diverse datasets. SpikePoint achieves state-of-the-art (SOTA) performance on four event-based action recognition datasets using only 16 timesteps, surpassing other SNN methods. Moreover, it also achieves SOTA performance across all methods on three datasets, utilizing approximately 0.3\% of the parameters and 0.5\% of power consumption employed by artificial neural networks (ANNs). These results emphasize the significance of Point Cloud and pave the way for many ultra-low-power event-based data processing applications.

翻译：事件相机是一种受生物启发的传感器，能够响应光强度的局部变化，具有低延迟、高能效和高动态范围的特点。同时，脉冲神经网络（SNNs）因其卓越的效率和容错性而备受关注。通过协同利用事件相机的固有能效和SNNs的脉冲处理能力，两者结合可实现超低功耗应用场景，例如动作识别任务。然而，现有方法通常将异步事件转换为传统帧格式，这不仅增加了额外的数据映射开销，还丧失了对稀疏性的保持，这与SNNs和事件相机的设计理念相悖。为解决这一挑战，我们提出SpikePoint——一种新颖的端到端点基SNN架构。SpikePoint擅长处理稀疏事件云数据，通过单阶段结构有效提取全局和局部特征。借助代理训练方法，SpikePoint以极少参数实现了高精度，并保持了低功耗特性，尤其在不同数据集的恒等映射特征提取器上表现突出。SpikePoint仅使用16个时间步便在四个基于事件的动作识别数据集上达到了最先进的性能，超越了其他SNN方法。此外，在三个数据集上，它实现了对所有方法的性能领先，其参数量仅为人工神经网络（ANNs）的约0.3%，功耗仅为约0.5%。这些结果强调了点云的重要性，并为众多超低功耗事件基数据处理应用铺平了道路。