Vision-as-inverse-graphics, the concept of reconstructing an image as an editable graphics program is a long-standing goal of computer vision. Yet even strong VLMs aren't able to achieve this in one-shot as they lack fine-grained spatial and physical grounding capability. Our key insight is that closing this gap requires interleaved multimodal reasoning through iterative execution and verification. Stemming from this, we present VIGA (Vision-as-Inverse-Graphic Agent) that starts from an empty world and reconstructs or edits scenes through a closed-loop write-run-render-compare-revise procedure. To support long-horizon reasoning, VIGA combines (i) a skill library that alternates generator and verifier roles and (ii) an evolving context memory that contains plans, code diffs, and render history. VIGA is task-agnostic as it doesn't require auxiliary modules, covering a wide range of tasks such as 3D reconstruction, multi-step scene editing, 4D physical interaction, and 2D document editing, etc. Empirically, we found VIGA substantially improves one-shot baselines on BlenderGym (35.32%) and SlideBench (117.17%). Moreover, VIGA is also model-agnostic as it doesn't require finetuning, enabling a unified protocol to evaluate heterogeneous foundation VLMs. To better support this protocol, we introduce BlenderBench, a challenging benchmark that stress-tests interleaved multimodal reasoning with graphics engine, where VIGA improves by 124.70%.

翻译：视觉作为逆向图形学，即将图像重建为可编辑图形程序的概念，是计算机视觉领域的长期目标。然而，即使是强大的视觉语言模型（VLMs）也无法通过单次推理实现这一目标，因为它们缺乏细粒度的空间与物理基础能力。我们的核心洞见在于：弥合这一差距需要通过迭代执行与验证的交错多模态推理。基于此，我们提出了VIGA（视觉作为逆向图形学智能体），该智能体从空场景出发，通过“编写-运行-渲染-比较-修订”的闭环流程重建或编辑场景。为支持长程推理，VIGA融合了（i）交替承担生成器与验证器角色的技能库，以及（ii）包含规划方案、代码差分与渲染历史的动态上下文记忆。VIGA具有任务无关性，无需辅助模块即可覆盖广泛任务，包括三维重建、多步场景编辑、四维物理交互及二维文档编辑等。实验表明，VIGA在BlenderGym（提升35.32%）和SlideBench（提升117.17%）上显著超越单次推理基线。此外，VIGA还具有模型无关性，无需微调即可实现异构基础视觉语言模型的统一评估协议。为更好地支持该协议，我们提出了BlenderBench——一个通过图形引擎对交错多模态推理进行压力测试的挑战性基准，VIGA在该基准上实现了124.70%的性能提升。