Vision-and-Language Navigation (VLN), as a crucial research problem of Embodied AI, requires an embodied agent to navigate through complex 3D environments following natural language instructions. Recent research has highlighted the promising capacity of large language models (LLMs) in VLN by improving navigational reasoning accuracy and interpretability. However, their predominant use in an offline manner usually suffers from substantial domain gap between the VLN task and the LLM training corpus. This paper introduces a novel strategy called Navigational Chain-of-Thought (NavCoT), where we fulfill parameter-efficient in-domain training to enable self-guided navigational decision, leading to a significant mitigation of the domain gap in a cost-effective manner. Specifically, at each timestep, the LLM is prompted to forecast the navigational chain-of-thought by: 1) acting as a world model to imagine the next observation according to the instruction, 2) selecting the candidate observation that best aligns with the imagination, and 3) determining the action based on the reasoning from the prior steps. Through constructing formalized labels for training, the LLM can learn to generate desired and reasonable chain-of-thought outputs for improving the action decision. Experimental results across various training settings and popular VLN benchmarks (e.g., Room-to-Room (R2R), Room-across-Room (RxR), Room-for-Room (R4R)) show the significant superiority of NavCoT over the direct action prediction variants. Through simple parameter-efficient finetuning, our NavCoT outperforms a recent GPT4-based approach with ~7% relative improvement on the R2R dataset. We believe that NavCoT will help unlock more task-adaptive and scalable LLM-based embodied agents, which are helpful for developing real-world robotics applications. Code is available at https://github.com/expectorlin/NavCoT.

翻译：视觉与语言导航（VLN）作为具身智能的关键研究问题，要求具身智能体依据自然语言指令在复杂三维环境中导航。近期研究通过提升导航推理的准确性和可解释性，展示了大型语言模型（LLM）在VLN中的潜力。然而，其主流离线使用方式常因VLN任务与LLM训练语料之间的显著领域差异而受限。本文提出一种名为“导航思维链”（NavCoT）的新策略，通过参数高效的领域内训练实现自主导航决策，以经济高效的方式显著缓解领域差异问题。具体而言，在每个时间步，模型被诱导生成导航思维链：1) 充当世界模型根据指令想象下一观察结果，2) 选择与想象最匹配的候选观察，3) 基于先前步骤的推理确定行动。通过构建形式化训练标签，LLM可学习生成合理且符合预期的思维链输出以优化行动决策。跨不同训练设置及主流VLN基准（如Room-to-Room (R2R)、Room-across-Room (RxR)、Room-for-Room (R4R)）的实验结果表明，NavCoT相较于直接动作预测变体具有显著优势。通过简单的参数高效微调，我们的NavCoT在R2R数据集上比近期基于GPT4的方法实现约7%的相对性能提升。我们相信NavCoT将助力开发更具任务适应性且可扩展的基于LLM的具身智能体，从而推动真实世界机器人应用的发展。代码已开源：https://github.com/expectorlin/NavCoT。