Multimodal Large Language Models (MLLMs) have made impressive progress in connecting vision and language, but they still struggle with spatial understanding and viewpoint-aware reasoning. Recent efforts aim to augment the input representations with geometric cues rather than explicitly teaching models to reason in 3D space. We introduce Loc3R-VLM, a framework that equips 2D Vision-Language Models with advanced 3D understanding capabilities from monocular video input. Inspired by human spatial cognition, Loc3R-VLM relies on two joint objectives: global layout reconstruction to build a holistic representation of the scene structure, and explicit situation modeling to anchor egocentric perspective. These objectives provide direct spatial supervision that grounds both perception and language in a 3D context. To ensure geometric consistency and metric-scale alignment, we leverage lightweight camera pose priors extracted from a pre-trained 3D foundation model. Loc3R-VLM achieves state-of-the-art performance in language-based localization and outperforms existing 2D- and video-based approaches on situated and general 3D question-answering benchmarks, demonstrating that our spatial supervision framework enables strong 3D understanding. Project page: https://kevinqu7.github.io/loc3r-vlm

翻译：多模态大语言模型在连接视觉与语言方面取得了令人瞩目的进展，但在空间理解与视角感知推理方面仍存在困难。近期的工作旨在通过几何线索增强输入表征，而非明确教导模型在三维空间中进行推理。我们提出Loc3R-VLM框架，该框架赋予二维视觉语言模型从单目视频输入中理解三维空间的高级能力。受人类空间认知启发，Loc3R-VLM依赖两个联合目标：全局布局重建以构建场景结构的整体表征，以及显式情境建模以锚定自我中心视角。这些目标提供了直接的几何监督，使感知与语言皆植根于三维语境。为确保几何一致性与度量尺度对齐，我们利用从预训练三维基础模型中提取的轻量级相机姿态先验。Loc3R-VLM在基于语言的定位任务中达到最先进性能，并在情境化与通用三维问答基准上超越现有基于二维与视频的方法，证明我们的空间监督框架能够实现强大的三维理解。项目页面：https://kevinqu7.github.io/loc3r-vlm