Robotic scene understanding increasingly relies on Vision-Language Models (VLMs) to generate natural language descriptions of the environment. In this work, we systematically evaluate single-view object captioning for tabletop scenes captured by a robotic manipulator, introducing a controlled physical domain shift that contrasts real-world tools with geometrically similar 3D-printed counterparts that differ in texture, colour, and material. We benchmark a suite of state-of-the-art, locally deployable VLMs across multiple metrics to assess semantic alignment and factual grounding. Our results demonstrate that while VLMs describe common real-world objects effectively, performance degrades markedly on 3D-printed items despite their structurally familiar forms. We further expose critical vulnerabilities in standard evaluation metrics, showing that some fail to detect domain shifts entirely or reward fluent but factually incorrect captions. These findings highlight the limitations of deploying foundation models for embodied agents and the need for more robust architectures and evaluation protocols in physical robotic applications.

翻译：机器人场景理解日益依赖视觉语言模型（VLM）生成环境的自然语言描述。本研究系统评估了机械臂采集的桌面场景中单视角物体描述能力，通过引入受控物理域转移，对比真实工具与几何结构相似但纹理、颜色和材质存在差异的3D打印工具。我们基准测试了多种可本地部署的先进VLM，通过多维指标评估语义对齐与事实依据。结果表明，尽管VLM能有效描述常见真实物体，但在3D打印物体上性能显著下降，即使这些物体保持结构相似性。我们进一步揭示标准评估指标的关键缺陷：部分指标完全无法检测域转移，或对流畅但事实错误的描述给予高分。这些发现凸显了将基础模型应用于具身代理的局限性，以及物理机器人应用中更鲁棒架构和评估协议的需求。