Learning to navigate to an image-specified goal is an important but challenging task for autonomous systems. The agent is required to reason the goal location from where a picture is shot. Existing methods try to solve this problem by learning a navigation policy, which captures semantic features of the goal image and observation image independently and lastly fuses them for predicting a sequence of navigation actions. However, these methods suffer from two major limitations. 1) They may miss detailed information in the goal image, and thus fail to reason the goal location. 2) More critically, it is hard to focus on the goal-relevant regions in the observation image, because they attempt to understand observation without goal conditioning. In this paper, we aim to overcome these limitations by designing a Fine-grained Goal Prompting (FGPrompt) method for image-goal navigation. In particular, we leverage fine-grained and high-resolution feature maps in the goal image as prompts to perform conditioned embedding, which preserves detailed information in the goal image and guides the observation encoder to pay attention to goal-relevant regions. Compared with existing methods on the image-goal navigation benchmark, our method brings significant performance improvement on 3 benchmark datasets (i.e., Gibson, MP3D, and HM3D). Especially on Gibson, we surpass the state-of-the-art success rate by 8% with only 1/50 model size. Project page: https://xinyusun.github.io/fgprompt-pages

翻译：学习导航至图像指定的目标位置是自主系统面临的重要但具有挑战性的任务。智能体需要从拍摄照片的位置推理出目标地点。现有方法试图通过学习导航策略来解决此问题，该策略独立提取目标图像与观测图像的语义特征，最终融合二者以预测一系列导航动作。然而，这些方法存在两大局限性：1）可能遗漏目标图像中的细节信息，从而导致无法准确推理目标位置；2）更关键的是，由于在理解观测图像时缺乏目标条件约束，难以聚焦于与目标相关的区域。本文提出了一种细粒度目标提示（FGPrompt）方法以克服上述局限。具体而言，我们利用目标图像中的细粒度高分辨率特征图作为提示信息进行条件嵌入，既保留目标图像的细节信息，又引导观测图像编码器关注与目标相关的区域。在图像目标导航基准测试中，我们的方法在三个基准数据集（Gibson、MP3D和HM3D）上均取得了显著的性能提升。特别是在Gibson数据集上，我们仅用1/50的模型参数规模就实现了超越现有最优方法8%的成功率。项目页面：https://xinyusun.github.io/fgprompt-pages