Streaming perception is a fundamental task in autonomous driving that requires a careful balance between the latency and accuracy of the autopilot system. However, current methods for streaming perception are limited as they rely only on the current and adjacent two frames to learn movement patterns, which restricts their ability to model complex scenes, often leading to poor detection results. To address this limitation, we propose LongShortNet, a novel dual-path network that captures long-term temporal motion and integrates it with short-term spatial semantics for real-time perception. Our proposed LongShortNet is notable as it is the first work to extend long-term temporal modeling to streaming perception, enabling spatiotemporal feature fusion. We evaluate LongShortNet on the challenging Argoverse-HD dataset and demonstrate that it outperforms existing state-of-the-art methods with almost no additional computational cost.

翻译：流式感知是自动驾驶中的一项基础任务，需要在自动驾驶系统的延迟与精度之间进行谨慎平衡。然而，当前流式感知方法受限于仅利用当前帧及相邻两帧来学习运动模式，这限制了其对复杂场景的建模能力，常导致检测结果不佳。为解决此局限，我们提出LongShortNet——一种新颖的双路径网络，能够捕捉长期时间运动并将其与短期空间语义相结合，以实现实时感知。我们的LongShortNet尤为突出，因为它是首个将长期时间建模扩展至流式感知的研究，实现了时空特征融合。我们在具有挑战性的Argoverse-HD数据集上评估了LongShortNet，并证明其以几乎无额外计算成本的方式超越了现有最先进方法。