What information is sufficient to learn the full richness of human scene understanding? The distributional hypothesis holds that the statistical co-occurrence of language and images captures the conceptual knowledge underlying visual cognition. Vision-language models (VLMs) are trained on massive paired text-image corpora but lack embodied experience, making them an ideal test of the distributional hypothesis. We report two experiments comparing descriptions generated by 18 VLMs to those of over 2000 human observers across 15 high-level scene understanding tasks, spanning general knowledge, affordances, sensory experiences, affective responses, and future prediction. Because many tasks lack ground truth answers, we developed a Human-Calibrated Cosine Distance (HCD) metric that measures VLM output similarity to the distribution of human responses, scaled by within-human variability. In Experiment 1, VLMs approached human-level performance on general knowledge tasks, but showed a robust deficit for affordance tasks that resisted prompt engineering and did not improve with newer model releases. In Experiment 2, we tested six mechanistic hypotheses for explaining this affordance gap, finding that the deficit was structural rather than stylistic and was not resolved by providing explicit spatial information. Corpus analyses revealed that image captioning datasets contain sparse agent-addressed affordance language, consistent with Gricean accounts of why embodied knowledge may be systematically underrepresented in language. Together, these findings suggest that distributional learning from images and text is insufficient for affordance-based scene understanding, implying that some dimensions of human visual cognition may require the kind of agent-centered, three-dimensional experience that no photograph or caption can encode.

翻译：什么信息足以习得人类场景理解的完整丰富性？分布假设认为，语言与图像的统计共现捕捉了视觉认知背后的概念知识。视觉-语言模型（VLMs）虽经大规模配对文本-图像语料训练，却缺乏具身经验，使其成为检验分布假设的理想对象。我们报告了两项实验，比较18个VLM与2000多名人类观察者在15项高级场景理解任务中的描述生成，涵盖通用知识、可供性、感官体验、情感响应及未来预测。由于众多任务缺乏真实答案，我们开发了人类校准余弦距离（HCD）指标——通过人类内部变异性缩放，衡量VLM输出与人类响应分布的相似度。实验1中，VLM在通用知识任务上接近人类水平，但在可供性任务上呈现稳健缺陷：该缺陷既未因提示工程而改善，也未随新模型发布而消减。实验2中，我们测试了六项机制假说以解释该可供性差距，发现缺陷具有结构性而非风格性，且明确空间信息的提供未能解决该问题。语料分析显示，图像描述数据集中包含稀疏的以主体为导向的可供性语言，这与格莱斯式解释一致——即具身知识可能系统性地在语言中未得到充分表征。总体而言，这些发现表明基于图像与文本的分布学习不足以支撑可供性驱动的场景理解，暗示人类视觉认知的某些维度可能需要照片或文字说明都无法编码的主体中心化、三维化具身体验。