The Multi-Modal Large Language Model (MLLM) refers to an extension of the Large Language Model (LLM) equipped with the capability to receive and infer multi-modal data. Spatial awareness stands as one of the crucial abilities of MLLM, encompassing diverse skills related to understanding spatial relationships among objects and between objects and the scene area. Industries such as autonomous driving, smart healthcare, robotics, virtual, and augmented reality heavily demand MLLM's spatial awareness capabilities. However, there exists a noticeable gap between the current spatial awareness capabilities of MLLM and the requirements set by human needs. To address this issue, this paper proposes using more precise spatial position information between objects to guide MLLM in providing more accurate responses to user-related inquiries. Specifically, for a particular multi-modal task, we utilize algorithms for acquiring geometric spatial information and scene graphs to obtain relevant geometric spatial information and scene details of objects involved in the query. Subsequently, based on this information, we direct MLLM to address spatial awareness-related queries posed by the user. Extensive experiments were conducted in benchmarks such as MME, MM-Vet, and other multi-modal large language models. The experimental results thoroughly confirm the efficacy of the proposed method in enhancing the spatial awareness tasks and associated tasks of MLLM.

翻译：多模态大语言模型（MLLM）是指具备接收和推理多模态数据能力的大语言模型（LLM）的扩展。空间感知是MLLM的关键能力之一，涵盖理解物体之间以及物体与场景区域之间空间关系的多种技能。自动驾驶、智能医疗、机器人、虚拟现实和增强现实等行业对MLLM的空间感知能力有较高需求。然而，当前MLLM的空间感知能力与人类需求之间仍存在明显差距。为解决此问题，本文提出利用物体间更精确的空间位置信息来引导MLLM对用户相关问题提供更准确的响应。具体而言，针对特定多模态任务，我们采用几何空间信息获取算法和场景图，得到查询中涉及物体的相关几何空间信息与场景细节。随后，基于这些信息，引导MLLM处理用户提出的空间感知相关问题。我们在MME、MM-Vet等基准及其他多模态大语言模型上进行了大量实验。实验结果充分验证了所提方法在增强MLLM空间感知任务及相关任务方面的有效性。