Reasoning Video Object Segmentation is a challenging task, aiming at generating a mask sequence from an input video given a complex and implicit text query. While existing works finetune Multimodal Large Language Models (MLLM) for the task, they still fail in video inputs given complex temporally-sensitive queries, indicating their lack of temporal and spatial integration in complex scenarios. In this paper, we propose CoT-RVS, a novel framework employing the zero-shot Chain-of-Thought (CoT) capability of MLLM to address these complex challenges by temporal-semantic reasoning: CoT-RVS analyzes the visible objects within a given frame that possibly match the language query (semantic), and chooses a corresponding keyframe for each object that can be observed effortlessly among all frames (temporal). Notably, the CoT-RVS framework is training-free and compatible with closed-source MLLMs, which can be applied to Reasoning Video Instance Segmentation. Our framework's training-free feature further allows its extension to process online video streams, where the CoT is used at test time to update the object of interest when a better target starts to emerge and becomes visible. We conduct extensive experiments on video object segmentation with explicit and implicit queries. The results show that CoT-RVS significantly outperforms previous works in both cases, qualitatively and quantitatively.

翻译：推理视频目标分割是一项具有挑战性的任务，其目标是在给定复杂且隐含的文本查询条件下，从输入视频生成掩码序列。现有工作虽然针对该任务对多模态大语言模型（MLLM）进行了微调，但在面对复杂的时间敏感查询时，它们仍难以处理视频输入，这表明其在复杂场景中缺乏时空整合能力。本文提出CoT-RVS，这是一个新颖的框架，它利用MLLM的零样本思维链（CoT）能力，通过时序-语义推理来应对这些复杂挑战：CoT-RVS分析给定帧中可能与语言查询匹配的可见对象（语义层面），并为每个对象在所有帧中选择一个能够被轻松观察到的对应关键帧（时序层面）。值得注意的是，CoT-RVS框架无需训练，且与闭源MLLM兼容，可应用于推理视频实例分割任务。该框架的免训练特性进一步使其能够扩展至处理在线视频流，在测试时利用CoT来更新感兴趣的目标对象，当更优的目标开始出现并变得可见时。我们在显式和隐含查询条件下对视频目标分割进行了大量实验。结果表明，无论在定性还是定量评估中，CoT-RVS在两种情况下均显著优于先前的工作。