Vision-language models demonstrate unprecedented performance and generalization across a wide range of tasks and scenarios. Integrating these foundation models into robotic navigation systems opens pathways toward building general-purpose robots. Yet, evaluating these models' navigation capabilities remains constrained by costly real-world trials, overly simplified simulations, and limited benchmarks. We introduce NaviTrace, a high-quality Visual Question Answering benchmark where a model receives an instruction and embodiment type (human, legged robot, wheeled robot, bicycle) and must output a 2D navigation trace in image space. Across 1000 scenarios and more than 3000 expert traces, we systematically evaluate eight state-of-the-art VLMs using a newly introduced semantic-aware trace score. This metric combines Dynamic Time Warping distance, goal endpoint error, and embodiment-conditioned penalties derived from per-pixel semantics and correlates with human preferences. Our evaluation reveals consistent gap to human performance caused by poor spatial grounding and goal localization. NaviTrace establishes a scalable and reproducible benchmark for real-world robotic navigation. The benchmark and leaderboard can be found at https://leggedrobotics.github.io/navitrace_webpage/.

翻译：视觉语言模型在广泛的任务和场景中展现出前所未有的性能和泛化能力。将这些基础模型集成到机器人导航系统中，为构建通用机器人开辟了路径。然而，评估这些模型的导航能力仍受限于成本高昂的真实世界试验、过度简化的仿真以及有限的基准测试。我们提出了NaviTrace，一个高质量的视觉问答基准，其中模型接收一条指令和具身类型（人类、腿式机器人、轮式机器人、自行车），并必须在图像空间中输出二维导航轨迹。在1000个场景和超过3000条专家轨迹上，我们使用新提出的语义感知轨迹评分，系统评估了八种最先进的视觉语言模型。该指标结合了动态时间规整距离、目标端点误差以及从逐像素语义推导出的具身条件惩罚，并与人类偏好相关。我们的评估揭示了由于空间定位和目标定位能力不足导致的与人类性能之间的持续差距。NaviTrace为真实世界机器人导航建立了一个可扩展且可复现的基准。基准测试和排行榜可在 https://leggedrobotics.github.io/navitrace_webpage/ 找到。