The utilisation of event cameras represents an important and swiftly evolving trend aimed at addressing the constraints of traditional video systems. Particularly within the automotive domain, these cameras find significant relevance for their integration into embedded real-time systems due to lower latency and power consumption. One effective approach to ensure the necessary throughput and latency for event processing is through the utilisation of graph convolutional networks (GCNs). In this study, we introduce a custom EFGCN (Event-based FPGA-accelerated Graph Convolutional Network) designed with a series of hardware-aware optimisations tailored for PointNetConv,a graph convolution designed for point cloud processing. The proposed techniques result in up to 100-fold reduction in model size compared to Asynchronous Event-based GNN (AEGNN), one of the most recent works in the field, with a relatively small decrease in accuracy (2.9% for the N-Caltech101 classification task, 2.2% for the N-Cars classification task), thus following the TinyML trend. We implemented EFGCN on a ZCU104 SoC FPGA platform without any off-chip external memory resources, achieving a throughput of 13.3 million events per second (MEPS) and real-time partially asynchronous processing with low latency. Across multiple event-based classification benchmarks, our approach achieves competitive accuracy while providing state-of-the-art computational efficiency per event, small model size, and high scalability, customisability and resource efficiency. We publish both software and hardware source code in an open repository: https://github.com/vision-agh/gcnn-dvs-fpga.

翻译：事件相机的应用代表了应对传统视频系统局限性的重要且快速发展的趋势。尤其在汽车领域，这类相机因低延迟和低功耗特性，在嵌入式实时系统的集成中展现出重要价值。为确保事件处理所需的吞吐量和延迟，图卷积网络（GCN）是一种有效方法。本研究提出一种定制化的EFGCN（基于事件的FPGA加速图卷积网络），该网络采用一系列针对点云处理图卷积算子PointNetConv的硬件感知优化技术。与领域内最新成果之一——异步事件图神经网络（AEGNN）相比，所提技术使模型规模缩减高达100倍，同时精度下降较小（在N-Caltech101分类任务中下降2.9%，在N-Cars分类任务中下降2.2%），从而契合TinyML趋势。我们在ZCU104型SoC FPGA平台上部署EFGCN，无需任何片外存储器资源，实现每秒1330万事件（MEPS）的吞吐量及低延迟的实时部分异步处理。在多个基于事件的分类基准测试中，本方法在保持竞争性精度的同时，提供了每事件计算效率、小型模型规模、高可扩展性、高定制化及高资源效率等领先性能。我们将软件与硬件源代码公开发布于开放仓库：https://github.com/vision-agh/gcnn-dvs-fpga。