The ability to accurately locate and navigate to a specific object is a crucial capability for embodied agents that operate in the real world and interact with objects to complete tasks. Such object navigation tasks usually require large-scale training in visual environments with labeled objects, which generalizes poorly to novel objects in unknown environments. In this work, we present a novel zero-shot object navigation method, Exploration with Soft Commonsense constraints (ESC), that transfers commonsense knowledge in pre-trained models to open-world object navigation without any navigation experience nor any other training on the visual environments. First, ESC leverages a pre-trained vision and language model for open-world prompt-based grounding and a pre-trained commonsense language model for room and object reasoning. Then ESC converts commonsense knowledge into navigation actions by modeling it as soft logic predicates for efficient exploration. Extensive experiments on MP3D, HM3D, and RoboTHOR benchmarks show that our ESC method improves significantly over baselines, and achieves new state-of-the-art results for zero-shot object navigation (e.g., 225\% relative Success Rate improvement than CoW on MP3D).

翻译：摘要：在真实世界中操作并交互物体以完成任务的具身智能体，其关键能力在于准确定位并导航至特定目标。这类目标导航任务通常需要在标注物体的视觉环境中进行大规模训练，但该方法对未知环境中的新型物体泛化能力较差。本文提出一种新颖的零样本目标导航方法——基于软常识约束的探索（ESC），该方法将预训练模型中的常识知识迁移至开放世界目标导航，无需任何导航经验或对视觉环境的额外训练。首先，ESC利用预训练的视觉-语言模型实现开放世界提示驱动的目标定位，并借助预训练的常识语言模型进行房间与物体推理。随后，通过将常识知识建模为软逻辑谓词以生成高效探索的导航动作，从而将常识知识转化为导航行为。在MP3D、HM3D和RoboTHOR基准上的大量实验表明，ESC方法显著超越基线，并在零样本目标导航任务中达到新最优水平（例如在MP3D数据集上相较CoW取得225%的相对成功率提升）。