Optical flow provides information on relative motion that is an important component in many computer vision pipelines. Neural networks provide high accuracy optical flow, yet their complexity is often prohibitive for application at the edge or in robots, where efficiency and latency play crucial role. To address this challenge, we build on the latest developments in event-based vision and spiking neural networks. We propose a new network architecture, inspired by Timelens, that improves the state-of-the-art self-supervised optical flow accuracy when operated both in spiking and non-spiking mode. To implement a real-time pipeline with a physical event camera, we propose a methodology for principled model simplification based on activity and latency analysis. We demonstrate high speed optical flow prediction with almost two orders of magnitude reduced complexity while maintaining the accuracy, opening the path for real-time deployments.

翻译：光流提供了相对运动的信息，是许多计算机视觉流水线中的重要组成部分。神经网络能够实现高精度的光流估计，但其复杂性常常阻碍其在边缘设备或机器人中的应用，而效率和延迟在这些场景中至关重要。为解决这一挑战，我们基于事件视觉和脉冲神经网络的最新进展，提出了一种受Timelens启发的新型网络架构，该架构在脉冲和非脉冲模式下运行时均能提升自监督光流的当前最优精度。为在物理事件相机上实现实时流水线，我们提出了一种基于活动度和延迟分析的原则性模型简化方法。我们展示了高速光流预测，其复杂度降低了近两个数量级，同时保持了精度，从而为实时部署开辟了道路。