Despite the significant advancements in natural language processing capabilities demonstrated by large language models such as ChatGPT, their proficiency in comprehending and processing spatial information, especially within the domains of 2D and 3D route planning, remains notably underdeveloped. This paper investigates the inherent limitations of ChatGPT and similar models in spatial reasoning and navigation-related tasks, an area critical for applications ranging from autonomous vehicle guidance to assistive technologies for the visually impaired. In this paper, we introduce a novel evaluation framework complemented by a baseline dataset, meticulously crafted for this study. This dataset is structured around three key tasks: plotting spatial points, planning routes in two-dimensional (2D) spaces, and devising pathways in three-dimensional (3D) environments. We specifically developed this dataset to assess the spatial reasoning abilities of ChatGPT. Our evaluation reveals key insights into the model's capabilities and limitations in spatial understanding.

翻译：尽管以ChatGPT为代表的大语言模型在自然语言处理能力上取得了显著进展，但其对空间信息的理解与处理能力——尤其是在二维和三维路径规划领域——仍明显不足。本文研究了ChatGPT及类似模型在空间推理与导航相关任务中的固有限制，该领域对从自动驾驶导航到视障辅助技术等应用至关重要。我们提出了一个结合本研究专门设计的基础数据集的新型评估框架。该数据集围绕三项核心任务构建：空间点标注、二维空间路径规划以及三维环境路径设计。我们专门开发该数据集以评估ChatGPT的空间推理能力。研究结果揭示了该模型在空间理解方面的能力特性与局限性的关键见解。