Effectively retrieving, reasoning, and understanding multimodal information remains a critical challenge for agentic systems. Traditional Retrieval-augmented Generation (RAG) methods rely on linear interaction histories, which struggle to handle long-context tasks, especially those involving information-sparse yet token-heavy visual data in iterative reasoning scenarios. To bridge this gap, we introduce VimRAG, a framework tailored for multimodal Retrieval-augmented Reasoning across text, images, and videos. Inspired by our systematic study, we model the reasoning process as a dynamic directed acyclic graph that structures the agent states and retrieved multimodal evidence. Building upon this structured memory, we introduce a Graph-Modulated Visual Memory Encoding mechanism, with which the significance of memory nodes is evaluated via their topological position, allowing the model to dynamically allocate high-resolution tokens to pivotal evidence while compressing or discarding trivial clues. To implement this paradigm, we propose a Graph-Guided Policy Optimization strategy. This strategy disentangles step-wise validity from trajectory-level rewards by pruning memory nodes associated with redundant actions, thereby facilitating fine-grained credit assignment. Extensive experiments demonstrate that VimRAG consistently achieves state-of-the-art performance on diverse multimodal RAG benchmarks. The code is available at https://github.com/Alibaba-NLP/VRAG.

翻译：有效检索、推理和理解多模态信息仍然是智能体系统的关键挑战。传统的检索增强生成方法依赖于线性的交互历史，难以处理长上下文任务，尤其是在迭代推理场景中涉及信息稀疏但标记密集的视觉数据时。为弥补这一差距，我们提出了VimRAG，一个专为跨文本、图像和视频的多模态检索增强推理而设计的框架。受系统性研究的启发，我们将推理过程建模为动态有向无环图，用以结构化智能体状态和检索到的多模态证据。基于这种结构化记忆，我们引入了图调制视觉记忆编码机制，通过记忆节点的拓扑位置评估其重要性，使模型能够动态地将高分辨率标记分配给关键证据，同时压缩或丢弃次要线索。为实现这一范式，我们提出了图引导策略优化策略。该策略通过剪枝与冗余动作相关的记忆节点，将逐步有效性与轨迹级奖励解耦，从而促进细粒度的信用分配。大量实验表明，VimRAG在多种多模态检索增强生成基准测试中始终取得最先进的性能。代码发布于https://github.com/Alibaba-NLP/VRAG。