In the field of robotics and automation, navigation systems based on Large Language Models (LLMs) have recently shown impressive performance. However, the security aspects of these systems have received relatively less attention. This paper pioneers the exploration of vulnerabilities in LLM-based navigation models in urban outdoor environments, a critical area given the technology's widespread application in autonomous driving, logistics, and emergency services. Specifically, we introduce a novel Navigational Prompt Suffix (NPS) Attack that manipulates LLM-based navigation models by appending gradient-derived suffixes to the original navigational prompt, leading to incorrect actions. We conducted comprehensive experiments on an LLMs-based navigation model that employs various LLMs for reasoning. Our results, derived from the Touchdown and Map2Seq street-view datasets under both few-shot learning and fine-tuning configurations, demonstrate notable performance declines across three metrics in the face of both white-box and black-box attacks. These results highlight the generalizability and transferability of the NPS Attack, emphasizing the need for enhanced security in LLM-based navigation systems. As an initial countermeasure, we propose the Navigational Prompt Engineering (NPE) Defense strategy, concentrating on navigation-relevant keywords to reduce the impact of adversarial suffixes. While initial findings indicate that this strategy enhances navigational safety, there remains a critical need for the wider research community to develop stronger defense methods to effectively tackle the real-world challenges faced by these systems.

翻译：在机器人技术与自动化领域，基于大型语言模型（LLM）的导航系统近期展现出卓越性能，然而这些系统的安全性问题受到的关注相对较少。本文率先探索了城市户外环境中基于LLM导航模型的脆弱性——鉴于该技术在自动驾驶、物流和紧急服务中的广泛应用，这一研究具有关键意义。具体而言，我们提出了一种新型导航提示后缀（NPS）攻击方法，通过将梯度推导生成的后缀附加至原始导航提示，操纵基于LLM的导航模型产生错误动作。我们在采用多种LLM进行推理的导航模型上开展了全面实验，基于Touchdown和Map2Seq街景数据集，在少样本学习与微调配置下进行测试。实验结果表明，在白盒与黑盒攻击场景中，三个评估指标均出现显著性能下降。这些结果凸显了NPS攻击的泛化性与迁移性，强调亟需增强基于LLM导航系统的安全性。作为初步防御方案，我们提出了导航提示工程（NPE）防御策略，通过聚焦导航相关关键词来削弱对抗性后缀的影响。尽管初步实验表明该策略能提升导航安全性，但学术界仍需开发更强大的防御方法以有效应对这些系统在现实世界中面临的挑战。