Spatio-temporal video grounding (or STVG) task aims at locating a spatio-temporal tube for a specific instance given a text query. Despite advancements, current methods easily suffer the distractors or heavy object appearance variations in videos due to insufficient object information from the text, leading to degradation. Addressing this, we propose a novel framework, context-guided STVG (CG-STVG), which mines discriminative instance context for object in videos and applies it as a supplementary guidance for target localization. The key of CG-STVG lies in two specially designed modules, including instance context generation (ICG), which focuses on discovering visual context information (in both appearance and motion) of the instance, and instance context refinement (ICR), which aims to improve the instance context from ICG by eliminating irrelevant or even harmful information from the context. During grounding, ICG, together with ICR, are deployed at each decoding stage of a Transformer architecture for instance context learning. Particularly, instance context learned from one decoding stage is fed to the next stage, and leveraged as a guidance containing rich and discriminative object feature to enhance the target-awareness in decoding feature, which conversely benefits generating better new instance context for improving localization finally. Compared to existing methods, CG-STVG enjoys object information in text query and guidance from mined instance visual context for more accurate target localization. In our experiments on three benchmarks, including HCSTVG-v1/-v2 and VidSTG, CG-STVG sets new state-of-the-arts in m_tIoU and m_vIoU on all of them, showing its efficacy. The code will be released at https://github.com/HengLan/CGSTVG.

翻译：时空视频定位（STVG）任务旨在根据文本描述在视频中定位特定目标的时空管状区域。尽管已有技术取得进展，现有方法仍易受视频中干扰因素或目标外观剧烈变化的影响，这源于文本信息无法提供充分的目标特征，导致性能下降。针对这一问题，我们提出一种新型框架——上下文引导的时空视频定位（CG-STVG），该框架挖掘视频中具有判别性的实例上下文信息，并将其作为目标定位的辅助线索。CG-STVG的核心在于两个专门设计的模块：实例上下文生成模块（ICG），专注于发现目标实例的外观与运动视觉上下文信息；以及实例上下文精炼模块（ICR），旨在通过剔除上下文中无关甚至有害的信息，提升ICG生成的实例上下文质量。在定位过程中，ICG与ICR协同部署于Transformer架构的每个解码阶段，用于学习实例上下文。特别地，前一解码阶段学习的实例上下文将传递至下一阶段，作为富含判别性目标特征的引导信息，增强解码特征中的目标感知能力，进而生成更优质的实例上下文，最终提升定位精度。相较于现有方法，CG-STVG同时利用文本查询中的目标信息与挖掘的实例视觉上下文作为引导，实现更精准的目标定位。在HCSTVG-v1/v2及VidSTG三个基准数据集上的实验表明，CG-STVG在m_tIoU与m_vIoU指标上均达到了新的最优性能。代码已开源：https://github.com/HengLan/CGSTVG。