Spatial perception and reasoning are crucial for Vision-Language-Action (VLA) models to accomplish fine-grained manipulation tasks. However, existing approaches often lack the ability to understand and reason over the essential 3D structures necessary for precise control. To address this limitation, we propose QDepth-VLA, a general framework that augments VLA models with an auxiliary depth prediction task. A dedicated depth expert is designed to predict quantized latent tokens of depth maps obtained from a VQ-VAE encoder, enabling the model to learn depth-aware representations that capture critical geometric cues. Experimental results on the simulation benchmarks and real-world tasks demonstrate that QDepth-VLA yields strong spatial reasoning and competitive performance on manipulation tasks.
翻译:空间感知与推理对于视觉-语言-动作(VLA)模型完成精细操作任务至关重要。然而,现有方法往往缺乏理解与推理精确控制所必需的三维结构能力。为解决这一局限性,我们提出QDepth-VLA——一种通用框架,通过引入辅助深度预测任务增强VLA模型。该框架设计了专门的深度专家模块,用于预测由VQ-VAE编码器获得深度图的量化潜变量标记,从而引导模型学习蕴含关键几何线索的深度感知表征。仿真基准测试与真实世界任务的实验结果表明,QDepth-VLA在操作任务中展现出强大的空间推理能力与竞争力。