While Multimodal Large Language Models (MLLMs) excel at visual understanding, they often struggle in complex scenarios that require visual planning and imagination. Inspired by how humans use sketching as a form of visual thinking to develop and communicate ideas, we introduce Latent Sketchpad, a framework that equips MLLMs with an internal visual scratchpad. The internal visual representations of MLLMs have traditionally been confined to perceptual understanding. We repurpose them to support generative visual thought without compromising reasoning ability. Building on frontier MLLMs, our approach integrates visual generation directly into their native autoregressive reasoning process. It allows the model to interleave textual reasoning with the generation of visual latents. These latents guide the internal thought process and can be translated into sketch images for interpretability. To realize this, we introduce two components: a Context-Aware Vision Head autoregressively produces visual representations, and a pretrained Sketch Decoder renders these into human-interpretable images. We evaluate the framework on our new dataset MazePlanning. Experiments across various MLLMs show that Latent Sketchpad delivers comparable or even superior reasoning performance to their backbone. It further generalizes across distinct frontier MLLMs, including Gemma3 and Qwen2.5-VL. By extending model's textual reasoning to visual thinking, our framework opens new opportunities for richer human-computer interaction and broader applications. More details and resources are available on our project page: https://latent-sketchpad.github.io/.

翻译：尽管多模态大语言模型（MLLMs）在视觉理解方面表现出色，但在需要视觉规划与想象的复杂场景中仍面临困难。受人类利用草图作为视觉思维形式来构思与表达想法的启发，我们提出了潜在画板框架，该框架为MLLMs配备了内部视觉草稿板。传统上，MLLMs的内部视觉表征主要局限于感知理解。我们重新定位其功能，使其在不损害推理能力的前提下支持生成式视觉思维。基于前沿MLLMs，我们的方法将视觉生成直接整合到其原生自回归推理过程中。该框架允许模型在文本推理过程中穿插生成视觉潜在表征。这些潜在表征引导内部思维过程，并可转换为草图图像以提升可解释性。为实现这一目标，我们引入了两个组件：上下文感知视觉头以自回归方式生成视觉表征，预训练草图解码器将这些表征渲染为人类可理解的图像。我们在新构建的数据集MazePlanning上评估该框架。针对多种MLLMs的实验表明，潜在画板在推理性能上达到甚至超越了其基础模型。该框架进一步实现了跨前沿MLLMs（包括Gemma3和Qwen2.5-VL）的泛化能力。通过将模型的文本推理扩展至视觉思维，我们的框架为更丰富的人机交互和更广泛的应用开辟了新机遇。更多细节与资源详见项目页面：https://latent-sketchpad.github.io/。