Vision-and-language navigation (VLN) enables the agent to navigate to a remote location following the natural language instruction in 3D environments. To represent the previously visited environment, most approaches for VLN implement memory using recurrent states, topological maps, or top-down semantic maps. In contrast to these approaches, we build the top-down egocentric and dynamically growing Grid Memory Map (i.e., GridMM) to structure the visited environment. From a global perspective, historical observations are projected into a unified grid map in a top-down view, which can better represent the spatial relations of the environment. From a local perspective, we further propose an instruction relevance aggregation method to capture fine-grained visual clues in each grid region. Extensive experiments are conducted on both the REVERIE, R2R, SOON datasets in the discrete environments, and the R2R-CE dataset in the continuous environments, showing the superiority of our proposed method.

翻译：摘要：视觉与语言导航（VLN）使智能体能够在三维环境中依据自然语言指令导航至远程位置。为表示先前访问过的环境，多数VLN方法采用循环状态、拓扑地图或自上而下的语义地图作为记忆。与这些方法不同，我们构建了自上而下的自我中心动态增长网格记忆地图（即GridMM），以结构化已访问环境。从全局视角，历史观测被投影到统一的俯视网格地图中，从而更好地表示环境的空间关系。从局部视角，我们进一步提出一种指令相关性聚合方法，以捕捉每个网格区域内的细粒度视觉线索。在离散环境中的REVERIE、R2R、SOON数据集以及连续环境中的R2R-CE数据集上进行了大量实验，结果表明了我们所提方法的优越性。