Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in vision-language understanding tasks. While these models often produce linguistically coherent output, they often suffer from hallucinations, generating descriptions that are factually inconsistent with the visual content, potentially leading to adverse consequences. Therefore, the assessment of hallucinations in MLLM has become increasingly crucial in the model development process. Contemporary methodologies predominantly depend on external LLM evaluators, which are themselves susceptible to hallucinations and may present challenges in terms of domain adaptation. In this study, we propose the hypothesis that hallucination manifests as measurable irregularities within the internal layer dynamics of MLLMs, not merely due to distributional shifts but also in the context of layer-wise analysis of specific assumptions. By incorporating such modifications, \textsc{\textsc{HalluShift++}} broadens the efficacy of hallucination detection from text-based large language models (LLMs) to encompass multimodal scenarios. Our codebase is available at https://github.com/C0mRD/HalluShift_Plus.

翻译：多模态大语言模型（MLLMs）在视觉-语言理解任务中展现出卓越的能力。尽管这些模型通常能生成语言连贯的输出，但它们常受幻觉问题困扰，即产生与视觉内容事实不一致的描述，可能导致不利后果。因此，在模型开发过程中，对MLLM中幻觉的评估变得日益关键。当前方法主要依赖外部LLM评估器，但这些评估器本身易受幻觉影响，且在领域适应方面可能存在挑战。在本研究中，我们提出假设：幻觉表现为MLLM内部层动态中可测量的异常，这不仅是由于分布偏移，还涉及对特定假设的逐层分析。通过引入此类修改，\\textsc{\\textsc{HalluShift++}}将幻觉检测的有效性从基于文本的大语言模型（LLMs）扩展到涵盖多模态场景。我们的代码库可在 https://github.com/C0mRD/HalluShift_Plus 获取。