Referring Video Object Segmentation (RVOS) aims to segment a target object throughout a video given a natural language query. Training-free methods for this task follow a common pipeline: a MLLM selects keyframes, grounds the referred object within those frames, and a video segmentation model propagates the results. While intuitive, this design asks the MLLM to make temporal decisions before any object-level evidence is available, limiting both reasoning quality and spatio-temporal coverage. To overcome this, we propose AgentRVOS, a training-free agentic pipeline built on the complementary strengths of SAM3 and a MLLM. Given a concept derived from the query, SAM3 provides reliable perception over the full spatio-temporal extent through generated mask tracks. The MLLM then identifies the target through query-grounded reasoning over this object-level evidence, iteratively pruning guided by SAM3's temporal existence information. Extensive experiments show that AgentRVOS achieves state-of-the-art performance among training-free methods across multiple benchmarks, with consistent results across diverse MLLM backbones. Our project page is available at: https://cvlab-kaist.github.io/AgentRVOS/.

翻译：指代视频目标分割（RVOS）旨在根据自然语言查询在整个视频中分割目标对象。该任务的免训练方法遵循通用流程：多模态大语言模型（MLLM）选择关键帧，定位所查询对象，再由视频分割模型传播结果。尽管直观，但这种设计要求MLLM在获取任何对象级证据之前做出时序决策，限制了推理质量与时空覆盖范围。为解决此问题，我们提出AgentRVOS——一种基于SAM3与MLLM互补优势的免训练代理流水线。给定从查询中提取的概念，SAM3通过生成的掩码轨迹提供完整的时空范围感知。随后，MLLM基于该对象级证据进行查询驱动的推理以识别目标，并利用SAM3的时序存在信息迭代修剪候选。大量实验表明，AgentRVOS在多个基准测试中均达到免训练方法的最优性能，且在不同MLLM骨干网络上结果一致。项目页面：https://cvlab-kaist.github.io/AgentRVOS/。