Despite the success of neural networks in computer vision tasks, digital 'neurons' are a very loose approximation of biological neurons. Today's learning approaches are designed to function on digital devices with digital data representations such as image frames. In contrast, biological vision systems are generally much more capable and efficient than state-of-the-art digital computer vision algorithms. Event cameras are an emerging sensor technology which imitates biological vision with asynchronously firing pixels, eschewing the concept of the image frame. To leverage modern learning techniques, many event-based algorithms are forced to accumulate events back to image frames, somewhat squandering the advantages of event cameras. We follow the opposite paradigm and develop a new type of neural network which operates closer to the original event data stream. We demonstrate state-of-the-art performance in angular velocity regression and competitive optical flow estimation, while avoiding difficulties related to training SNN. Furthermore, the processing latency of our proposed approach is less than 1/10 any other implementation, while continuous inference increases this improvement by another order of magnitude.

翻译：尽管神经网络在计算机视觉任务中取得了成功，但数字"神经元"对生物神经元的模拟仍非常粗略。当前的学习方法旨在使用图像帧等数字数据表示的数字设备上运行。相比之下，生物视觉系统通常比最先进的数字计算机视觉算法具有更强的能力和更高的效率。事件相机是一种新兴的传感器技术，它通过异步触发像素来模仿生物视觉，摒弃了图像帧的概念。为了利用现代学习技术，许多基于事件的算法被迫将事件重新累积为图像帧，这在一定程度上浪费了事件相机的优势。我们遵循相反的范式，开发了一种新型神经网络，使其更贴近原始事件数据流进行操作。我们在角速度回归任务中展示了最先进的性能，并在光流估计中取得了具有竞争力的结果，同时避免了训练脉冲神经网络（SNN）带来的困难。此外，我们提出的方法处理延迟低于其他任何实现的1/10，而连续推理可将这一优势再提升一个数量级。