Vision-Language Models (VLMs) have demonstrated remarkable progress in multimodal tasks, but remain susceptible to hallucinations, where generated text deviates from the underlying visual content. Existing hallucination detection methods primarily rely on output logits or external verification tools, often overlooking their internal mechanisms. In this work, we investigate the outputs of internal attention heads, postulating that specific heads carry the primary signals for truthful generation.However, directly probing these high-dimensional states is challenging due to the entanglement of visual-linguistic syntax and noise. To address this, we propose VIB-Probe, a novel hallucination detection and mitigation framework leveraging the Variational Information Bottleneck (VIB) theory. Our method extracts discriminative patterns across layers and heads while filtering out semantic nuisances through the information bottleneck principle. Furthermore, by leveraging the gradients of our VIB probe, we identify attention heads with strong causal influence on hallucinations and introduce an inference-time intervention strategy for hallucination mitigation. Extensive experiments across diverse benchmarks demonstrate that VIB-Probe significantly outperforms existing baselines in both settings. Our code will be made publicly available.

翻译：视觉语言模型（VLMs）在多模态任务中展现出显著进展，但仍易产生幻觉现象，即生成的文本偏离底层视觉内容。现有的幻觉检测方法主要依赖于输出逻辑值或外部验证工具，往往忽视了其内部机制。本研究通过分析内部注意力头的输出，提出假设：特定注意力头承载着真实性生成的主要信号。然而，由于视觉-语言句法与噪声的纠缠，直接探测这些高维状态具有挑战性。为解决此问题，我们提出VIB-Probe——一种基于变分信息瓶颈（VIB）理论的新型幻觉检测与缓解框架。该方法通过信息瓶颈原理，在提取跨层跨注意力头的判别性模式的同时过滤语义干扰。进一步地，通过利用VIB探针的梯度信息，我们识别出对幻觉具有强因果影响的注意力头，并提出一种推理时干预策略以缓解幻觉。在多样化基准测试上的大量实验表明，VIB-Probe在两种设置下均显著优于现有基线方法。我们的代码将公开提供。