Vision-Language Models (VLMs) achieve strong results on multimodal tasks such as visual question answering, yet they can still fail even when the correct visual evidence is present. In this work, we systematically investigate whether these failures arise from not perceiving the evidence or from not leveraging it effectively. By examining layer-wise attention dynamics, we find that shallow layers focus primarily on text, while deeper layers sparsely but reliably attend to localized evidence regions. Surprisingly, VLMs often perceive the visual evidence when outputting incorrect answers, a phenomenon we term ``seeing but not believing'' that widely exists in major VLM families. Building on this, we introduce an inference-time intervention that highlights deep-layer evidence regions through selective attention-based masking. It requires no training and consistently improves accuracy across multiple families, including LLaVA, Qwen, Gemma, and InternVL. These results show that VLMs encode reliable evidence internally but under-utilize it, making such signals explicit can bridge the gap between perception and reasoning, advancing the diagnostic understanding and reliability of VLMs.

翻译：视觉语言模型（VLMs）在视觉问答等多模态任务上表现出色，但它们仍可能在正确的视觉证据存在时给出错误答案。本研究系统性地探究这些失败源于未能感知证据，还是未能有效利用证据。通过分析层级注意力动态，我们发现浅层主要关注文本，而深层虽稀疏但可靠地关注局部证据区域。令人惊讶的是，VLMs在输出错误答案时往往已感知到视觉证据，这一现象我们称之为“眼见未必为信”，且广泛存在于主流VLM系列中。基于此，我们提出一种推理时干预方法，通过基于注意力的选择性掩码来突出深层证据区域。该方法无需训练，并在包括LLaVA、Qwen、Gemma和InternVL在内的多个模型系列中持续提升准确率。这些结果表明，VLMs在内部编码了可靠的证据但未能充分利用，使此类信号显式化可以弥合感知与推理之间的差距，从而推进对VLMs的诊断性理解与可靠性提升。