This paper explores the synergistic potential of neuromorphic and edge computing to create a versatile machine learning (ML) system tailored for processing data captured by dynamic vision sensors. We construct and train hybrid models, blending spiking neural networks (SNNs) and artificial neural networks (ANNs) using PyTorch and Lava frameworks. Our hybrid architecture integrates an SNN for temporal feature extraction and an ANN for classification. We delve into the challenges of deploying such hybrid structures on hardware. Specifically, we deploy individual components on Intel's Neuromorphic Processor Loihi (for SNN) and Jetson Nano (for ANN). We also propose an accumulator circuit to transfer data from the spiking to the non-spiking domain. Furthermore, we conduct comprehensive performance analyses of hybrid SNN-ANN models on a heterogeneous system of neuromorphic and edge AI hardware, evaluating accuracy, latency, power, and energy consumption. Our findings demonstrate that the hybrid spiking networks surpass the baseline ANN model across all metrics and outperform the baseline SNN model in accuracy and latency.

翻译：本文探讨了神经形态计算与边缘计算的协同潜力，旨在构建一个专为处理动态视觉传感器捕获数据而设计的通用机器学习系统。我们利用PyTorch和Lava框架构建并训练了融合脉冲神经网络与人工神经网络的混合模型。该混合架构集成了用于时序特征提取的SNN和用于分类的ANN。我们深入研究了此类混合结构在硬件部署中的挑战，具体将各组件分别部署于英特尔的神经形态处理器Loihi（运行SNN）和Jetson Nano（运行ANN）。同时，我们提出了一种累加器电路来实现从脉冲域到非脉冲域的数据传输。此外，我们在神经形态与边缘AI硬件构成的异构系统上对混合SNN-ANN模型进行了全面的性能分析，评估了其精度、延迟、功耗和能耗。研究结果表明，混合脉冲网络在所有指标上均超越基准ANN模型，并在精度和延迟方面优于基准SNN模型。