Efficient trajectory planning in off-road terrains presents a formidable challenge for autonomous vehicles, often necessitating complex multi-step pipelines. However, traditional approaches exhibit limited adaptability in dynamic environments. To address these limitations, this paper proposes OFF-EMMA, a novel end-to-end multimodal framework designed to overcome the deficiencies of insufficient spatial perception and unstable reasoning in visual-language-action (VLA) models for off-road autonomous driving scenarios. The framework explicitly annotates input images through the design of a visual prompt block and introduces a chain-of-thought with self-consistency (COT-SC) reasoning strategy to enhance the accuracy and robustness of trajectory planning. The visual prompt block utilizes semantic segmentation masks as visual prompts, enhancing the spatial understanding ability of pre-trained visual-language models for complex terrains. The COT- SC strategy effectively mitigates the error impact of outliers on planning performance through a multi-path reasoning mechanism. Experimental results on the RELLIS-3D off-road dataset demonstrate that OFF-EMMA significantly outperforms existing methods, reducing the average L2 error of the Qwen backbone model by 13.3% and decreasing the failure rate from 16.52% to 6.56%.

翻译：越野地形中的高效轨迹规划对自动驾驶车辆提出了严峻挑战，通常需要复杂的多步骤流程。然而，传统方法在动态环境中的适应性有限。为应对这些局限性，本文提出了OFF-EMMA——一种新颖的端到端多模态框架，旨在克服越野自动驾驶场景中视觉-语言-动作模型存在的空间感知不足与推理不稳定的缺陷。该框架通过设计视觉提示块对输入图像进行显式标注，并引入具有自洽性的思维链推理策略，以提升轨迹规划的准确性与鲁棒性。视觉提示块利用语义分割掩码作为视觉提示，增强了预训练视觉-语言模型对复杂地形的空间理解能力。COT-SC策略通过多路径推理机制，有效降低了异常值对规划性能的误差影响。在RELLIS-3D越野数据集上的实验结果表明，OFF-EMMA显著优于现有方法，将Qwen骨干模型的平均L2误差降低了13.3%，并将失败率从16.52%降至6.56%。