This paper aims to tackle a novel task - Temporal Sentence Grounding in Streaming Videos (TSGSV). The goal of TSGSV is to evaluate the relevance between a video stream and a given sentence query. Unlike regular videos, streaming videos are acquired continuously from a particular source, and are always desired to be processed on-the-fly in many applications such as surveillance and live-stream analysis. Thus, TSGSV is challenging since it requires the model to infer without future frames and process long historical frames effectively, which is untouched in the early methods. To specifically address the above challenges, we propose two novel methods: (1) a TwinNet structure that enables the model to learn about upcoming events; and (2) a language-guided feature compressor that eliminates redundant visual frames and reinforces the frames that are relevant to the query. We conduct extensive experiments using ActivityNet Captions, TACoS, and MAD datasets. The results demonstrate the superiority of our proposed methods. A systematic ablation study also confirms their effectiveness.

翻译：本文旨在解决一项新颖任务——流式视频中的时间语句定位（TSGSV）。TSGSV的目标是评估视频流与给定语句查询之间的相关性。与常规视频不同，流式视频从特定源连续获取，且在监控和直播分析等应用中通常需要实时处理。因此，TSGSV具有挑战性，因为它要求模型在缺少未来帧的情况下进行推理，并有效处理长历史帧，而这在早期方法中尚未涉及。为专门应对上述挑战，我们提出两种新颖方法：（1）双网络结构使模型能够学习即将发生的事件；（2）语言引导的特征压缩器可消除冗余视觉帧并增强与查询相关的帧。我们使用ActivityNet Captions、TACoS和MAD数据集进行了大量实验，结果表明所提方法具有优越性。系统的消融研究也证实了其有效性。