Multi-modal Large Language Models (MLLMs) have demonstrated strong capabilities in general-purpose perception and reasoning, but they still struggle with tasks that require spatial understanding of the 3D world. To address this, we introduce pySpatial, a visual programming framework that equips MLLMs with the ability to interface with spatial tools via Python code generation. Given an image sequence and a natural-language query, the model composes function calls to spatial tools including 3D reconstruction, camera-pose recovery, novel-view rendering, etc. These operations convert raw 2D inputs into an explorable 3D scene, enabling MLLMs to reason explicitly over structured spatial representations. Notably, pySpatial requires no gradient-based fine-tuning and operates in a fully zero-shot setting. Experimental evaluations on the challenging MindCube and Omni3D-Bench benchmarks demonstrate that our framework pySpatial consistently surpasses strong MLLM baselines; for instance, it outperforms GPT-4.1-mini by 12.94% on MindCube. Furthermore, we conduct real-world indoor navigation experiments where the robot can successfully traverse complex environments using route plans generated by pySpatial, highlighting the practical effectiveness of our approach.

翻译：多模态大语言模型（MLLMs）已在通用感知与推理任务中展现出强大能力，但在需要理解三维世界空间关系的任务上仍存在困难。为此，我们提出了pySpatial——一种视觉编程框架，它通过生成Python代码使MLLMs能够与空间工具进行交互。给定图像序列和自然语言查询，该模型可组合调用包括三维重建、相机位姿恢复、新视角渲染等在内的空间工具函数。这些操作将原始二维输入转换为可探索的三维场景，使MLLMs能够基于结构化的空间表征进行显式推理。值得注意的是，pySpatial无需基于梯度的微调，完全在零样本设置下运行。在具有挑战性的MindCube和Omni3D-Bench基准测试上的实验评估表明，我们的框架pySpatial持续超越强大的MLLM基线模型；例如，其在MindCube上以12.94%的优势超过GPT-4.1-mini。此外，我们在真实室内导航实验中验证了该框架的实用性：机器人能够利用pySpatial生成的路径规划成功穿越复杂环境，这凸显了我们方法的实际有效性。