The application of vision-language models (VLMs) has achieved impressive success in various robotics tasks. However, there are few explorations for these foundation models used in quadruped robot navigation through terrains in 3D environments. In this work, we introduce SARO (Space Aware Robot System for Terrain Crossing), an innovative system composed of a high-level reasoning module, a closed-loop sub-task execution module, and a low-level control policy. It enables the robot to navigate across 3D terrains and reach the goal position. For high-level reasoning and execution, we propose a novel algorithmic system taking advantage of a VLM, with a design of task decomposition and a closed-loop sub-task execution mechanism. For low-level locomotion control, we utilize the Probability Annealing Selection (PAS) method to effectively train a control policy by reinforcement learning. Numerous experiments show that our whole system can accurately and robustly navigate across several 3D terrains, and its generalization ability ensures the applications in diverse indoor and outdoor scenarios and terrains. Project page: https://saro-vlm.github.io/

翻译：视觉语言模型（VLM）在各类机器人任务中的应用已取得显著成功。然而，这些基础模型在四足机器人三维环境地形导航中的应用探索仍较为有限。本研究提出SARO（空间感知机器人地形穿越系统），这是一个由高层推理模块、闭环子任务执行模块以及底层控制策略组成的创新系统。该系统使机器人能够在三维地形中导航并抵达目标位置。针对高层推理与执行，我们提出一种新颖的算法系统，该系统利用视觉语言模型的优势，设计了任务分解机制与闭环子任务执行架构。在底层运动控制方面，我们采用概率退火选择方法，通过强化学习高效训练控制策略。大量实验表明，我们的完整系统能够在多种三维地形中实现精准鲁棒的导航，其泛化能力确保了在多样化的室内外场景与地形中的应用潜力。项目主页：https://saro-vlm.github.io/