It is natural to construct a multi-frame instead of a single-frame 3D detector for a continuous-time stream. Although increasing the number of frames might improve performance, previous multi-frame studies only used very limited frames to build their systems due to the dramatically increased computational and memory cost. To address these issues, we propose a novel on-stream training and prediction framework that, in theory, can employ an infinite number of frames while keeping the same amount of computation as a single-frame detector. This infinite framework (INT), which can be used with most existing detectors, is utilized, for example, on the popular CenterPoint, with significant latency reductions and performance improvements. We've also conducted extensive experiments on two large-scale datasets, nuScenes and Waymo Open Dataset, to demonstrate the scheme's effectiveness and efficiency. By employing INT on CenterPoint, we can get around 7% (Waymo) and 15% (nuScenes) performance boost with only 2~4ms latency overhead, and currently SOTA on the Waymo 3D Detection leaderboard.

翻译：将连续时间流中的多帧而非单帧三维检测器视为一种自然构想。尽管增加帧数可能提升性能，但先前多帧研究因计算和内存成本急剧上升而仅采用极有限帧数构建系统。为解决这些问题，我们提出一种新颖的流式训练与预测框架，理论上可在保持与单帧检测器相同计算量的同时，利用无限帧数。这一无限框架（INT）可适配大多数现有检测器，例如在流行的CenterPoint上应用时，显著降低了延迟并提升了性能。我们还在nuScenes和Waymo Open Dataset两个大规模数据集上开展了广泛实验，以验证该方案的有效性与高效性。通过在CenterPoint上应用INT，我们仅需2-4毫秒的延迟开销即可获得约7%（Waymo）和15%（nuScenes）的性能提升，并达到当前Waymo三维检测排行榜的领先水平。