Event cameras have the ability to record continuous and detailed trajectories of objects with high temporal resolution, thereby providing intuitive motion cues for optical flow estimation. Nevertheless, most existing learning-based approaches for event optical flow estimation directly remould the paradigm of conventional images by representing the consecutive event stream as static frames, ignoring the inherent temporal continuity of event data. In this paper, we argue that temporal continuity is a vital element of event-based optical flow and propose a novel Temporal Motion Aggregation (TMA) approach to unlock its potential. Technically, TMA comprises three components: an event splitting strategy to incorporate intermediate motion information underlying the temporal context, a linear lookup strategy to align temporally fine-grained motion features and a novel motion pattern aggregation module to emphasize consistent patterns for motion feature enhancement. By incorporating temporally fine-grained motion information, TMA can derive better flow estimates than existing methods at early stages, which not only enables TMA to obtain more accurate final predictions, but also greatly reduces the demand for a number of refinements. Extensive experiments on DSEC-Flow and MVSEC datasets verify the effectiveness and superiority of our TMA. Remarkably, compared to E-RAFT, TMA achieves a 6\% improvement in accuracy and a 40\% reduction in inference time on DSEC-Flow. Code will be available at \url{https://github.com/ispc-lab/TMA}.
翻译:事件相机能够以高时间分辨率记录物体连续且精细的运动轨迹,从而为光流估计提供直观的运动线索。然而,现有大多数基于学习的事件光流估计方法直接将传统图像的范式移植过来,将连续的事件流表示为静态帧,忽视了事件数据固有的时域连续性。本文指出,时域连续性是事件光流的关键要素,并提出一种新颖的时域运动聚合方法(TMA)以挖掘其潜力。技术上,TMA包含三个组件:一种事件分割策略,用于融会时域上下文中的中间运动信息;一种线性查找策略,用于对齐时域细粒度运动特征;以及一种新型运动模式聚合模块,用于强化一致模式以增强运动特征。通过融会时域细粒度运动信息,TMA在早期阶段即可获得比现有方法更优的流估计,这不仅使其最终预测更加精确,还大幅减少了对多次优化的需求。在DSEC-Flow和MVSEC数据集上的大量实验验证了TMA的有效性与优越性。值得注意的是,与E-RAFT相比,TMA在DSEC-Flow上准确率提升6%,推理时间减少40%。代码将在 \url{https://github.com/ispc-lab/TMA} 开源。