Video reasoning constitutes a comprehensive assessment of a model's capabilities, as it demands robust perceptual and interpretive skills, thereby serving as a means to explore the boundaries of model performance. While recent research has leveraged text-centric Chain-of-Thought reasoning to augment these capabilities, such approaches frequently suffer from representational mismatch and restricted by limited perceptual acuity. To address these limitations, we propose Weaver, a novel, end-to-end trainable multimodal reasoning agentic system. Weaver empowers its policy model to dynamically invoke diverse tools throughout the reasoning process, enabling progressive acquisition of crucial visual cues and construction of authentic multimodal reasoning trajectories. Furthermore, we integrate a reinforcement learning algorithm to allow the system to freely explore strategies for employing and combining these tools with trajectory-free data. Extensive experiments demonstrate that our system, Weaver, enhances performance on several complex video reasoning benchmarks, particularly those involving long videos.
翻译:视频推理构成了对模型能力的全面评估,因为它要求模型具备强大的感知与解释能力,从而可作为探索模型性能边界的有效手段。尽管近期研究利用以文本为中心的思维链推理来增强这些能力,但此类方法常受表征失配问题困扰,且受限于有限的感知敏锐度。为应对这些局限,我们提出了Weaver,一种新颖的、可端到端训练的多模态推理智能体系统。Weaver使其策略模型能够在推理过程中动态调用多样化工具,从而实现关键视觉线索的渐进式获取与真实多模态推理轨迹的构建。此外,我们引入强化学习算法,使系统能够自由探索使用和组合这些工具的策略,并利用无轨迹数据进行训练。大量实验表明,我们的Weaver系统在多个复杂视频推理基准测试中提升了性能,尤其是在涉及长视频的任务上表现突出。