Visual target navigation in unknown environments is a crucial problem in robotics. Despite extensive investigation of classical and learning-based approaches in the past, robots lack common-sense knowledge about household objects and layouts. Prior state-of-the-art approaches to this task rely on learning the priors during the training and typically require significant expensive resources and time for learning. To address this, we propose a new framework for visual target navigation that leverages Large Language Models (LLM) to impart common sense for object searching. Specifically, we introduce two paradigms: (i) zero-shot and (ii) feed-forward approaches that use language to find the relevant frontier from the semantic map as a long-term goal and explore the environment efficiently. Our analysis demonstrates the notable zero-shot generalization and transfer capabilities from the use of language. Experiments on Gibson and Habitat-Matterport 3D (HM3D) demonstrate that the proposed framework significantly outperforms existing map-based methods in terms of success rate and generalization. Ablation analysis also indicates that the common-sense knowledge from the language model leads to more efficient semantic exploration. Finally, we provide a real robot experiment to verify the applicability of our framework in real-world scenarios. The supplementary video and code can be accessed via the following link: https://sites.google.com/view/l3mvn.

翻译：在未知环境中进行视觉目标导航是机器人学中的一个关键问题。尽管过去对经典方法和基于学习的方法进行了广泛研究，但机器人在关于家庭物品和布局方面缺乏常识性知识。此前针对该任务的先进方法依赖于训练过程中学习先验知识，通常需要大量昂贵的资源和时间进行学习。为了解决这一问题，我们提出了一种新的视觉目标导航框架，利用大型语言模型（LLM）赋予物体搜索的常识性能力。具体而言，我们引入了两种范式：（i）零样本方法和（ii）前馈方法，这些方法使用语言从语义地图中找到相关的前沿区域作为长期目标，并高效探索环境。我们的分析证明了使用语言所带来的显著零样本泛化和迁移能力。在Gibson和Habitat-Matterport 3D（HM3D）上的实验表明，所提出的框架在成功率和泛化性方面显著优于现有的基于地图的方法。消融分析也表明，语言模型中的常识知识能够带来更高效的语义探索。最后，我们提供了一项真实机器人实验，以验证该框架在现实场景中的适用性。补充视频和代码可通过以下链接获取：https://sites.google.com/view/l3mvn。