Optical flow provides information on relative motion that is an important component in many computer vision pipelines. Neural networks provide high accuracy optical flow, yet their complexity is often prohibitive for application at the edge or in robots, where efficiency and latency play crucial role. To address this challenge, we build on the latest developments in event-based vision and spiking neural networks. We propose a new network architecture, inspired by Timelens, that improves the state-of-the-art self-supervised optical flow accuracy when operated both in spiking and non-spiking mode. To implement a real-time pipeline with a physical event camera, we propose a methodology for principled model simplification based on activity and latency analysis. We demonstrate high speed optical flow prediction with almost two orders of magnitude reduced complexity while maintaining the accuracy, opening the path for real-time deployments.

翻译：光流提供相对运动信息，是许多计算机视觉流程中的重要组成部分。神经网络能实现高精度光流估计，但其复杂度往往阻碍其在边缘端或机器人中的应用——这些场景对效率和延迟有严苛要求。为应对这一挑战，我们基于事件视觉和脉冲神经网络的最新进展，提出了一种受Timelens启发的新型网络架构。该架构在脉冲与非脉冲模式下均能提升当前最先进的自监督光流精度。为实现物理事件相机的实时流水线，我们提出了一种基于活跃度和延迟分析的原则性模型简化方法。实验表明，我们能在保持精度的情况下，将复杂度降低近两个数量级，实现高速光流预测，为实时部署开辟了道路。