Consecutive frames in a video contain redundancy, but they may also contain relevant complementary information for the detection task. The objective of our work is to leverage this complementary information to improve detection. Therefore, we propose a spatio-temporal fusion framework (STF). We first introduce multi-frame and single-frame attention modules that allow a neural network to share feature maps between nearby frames to obtain more robust object representations. Second, we introduce a dual-frame fusion module that merges feature maps in a learnable manner to improve them. Our evaluation is conducted on three different benchmarks including video sequences of moving road users. The performed experiments demonstrate that the proposed spatio-temporal fusion module leads to improved detection performance compared to baseline object detectors. Code is available at https://github.com/noreenanwar/STF-module

翻译：视频中连续帧包含冗余信息，但同时也可能包含对检测任务有用的互补信息。我们的工作旨在利用这些互补信息来提升检测性能。为此，我们提出了一种时空融合框架（STF）。首先，我们引入了多帧和单帧注意力模块，使神经网络能够在邻近帧之间共享特征图，从而获得更鲁棒的物体表示。其次，我们引入了一个双帧融合模块，该模块以可学习的方式融合特征图以增强其表达。我们在包括移动道路使用者视频序列在内的三个不同基准数据集上进行了评估。实验结果表明，与基线目标检测器相比，所提出的时空融合模块能够显著提升检测性能。代码发布于https://github.com/noreenanwar/STF-module。