Mobile robots are often deployed over long durations in diverse open, dynamic scenes, including indoor setting such as warehouses and manufacturing facilities, and outdoor settings such as agricultural and roadway operations. A core challenge is to build a scalable long-horizon memory that supports an agentic workflow for planning, retrieval, and reasoning over open-ended instructions at variable granularity, while producing precise, actionable answers for navigation. We present STaR, an agentic reasoning framework that (i) constructs a task-agnostic, multimodal long-term memory that generalizes to unseen queries while preserving fine-grained environmental semantics (object attributes, spatial relations, and dynamic events), and (ii) introduces a Scalable Task Conditioned Retrieval algorithm based on the Information Bottleneck principle to extract from long-term memory a compact, non-redundant, information-rich set of candidate memories for contextual reasoning. We evaluate STaR on NaVQA (mixed indoor/outdoor campus scenes) and WH-VQA, a customized warehouse benchmark with many visually similar objects built with Isaac Sim, emphasizing contextual reasoning. Across the two datasets, STaR consistently outperforms strong baselines, achieving higher success rates and markedly lower spatial error. We further deploy STaR on a real Husky wheeled robot in both indoor and outdoor environments, demonstrating robust long horizon reasoning, scalability, and practical utility. Project Website: https://trailab.github.io/STaR-website/
翻译:移动机器人常需在多样化的开放动态场景中长期部署,包括仓库与制造设施等室内环境,以及农业与道路作业等室外环境。其核心挑战在于构建可扩展的长时程记忆系统,以支持基于开放式多粒度指令的自主工作流,实现规划、检索与推理,并生成精确、可执行的导航决策。本文提出STaR——一种自主推理框架,其具备以下特点:(i)构建任务无关的多模态长期记忆,该记忆能泛化至未见查询,同时保留细粒度环境语义(物体属性、空间关系及动态事件);(ii)基于信息瓶颈原理提出可扩展任务条件检索算法,从长期记忆中提取紧凑、非冗余、信息丰富的候选记忆集以支持上下文推理。我们在NaVQA(室内外混合校园场景)与WH-VQA(基于Isaac Sim构建的、包含大量视觉相似物体的定制化仓库基准数据集)上评估STaR,重点关注上下文推理能力。在两个数据集中,STaR均持续超越现有强基线方法,获得更高的任务成功率与显著降低的空间误差。我们进一步在室内外环境中将STaR部署于真实的Husky轮式机器人,验证了其具备鲁棒的长时程推理能力、可扩展性及实际应用价值。项目网站:https://trailab.github.io/STaR-website/