Large Vision-Language Models (LVLMs) have recently achieved remarkable success. However, LVLMs are still plagued by the hallucination problem, which limits the practicality in many scenarios. Hallucination refers to the information of LVLMs' responses that does not exist in the visual input, which poses potential risks of substantial consequences. There has been limited work studying hallucination evaluation in LVLMs. In this paper, we propose Hallucination Evaluation based on Large Language Models (HaELM), an LLM-based hallucination evaluation framework. HaELM achieves an approximate 95% performance comparable to ChatGPT and has additional advantages including low cost, reproducibility, privacy preservation and local deployment. Leveraging the HaELM, we evaluate the hallucination in current LVLMs. Furthermore, we analyze the factors contributing to hallucination in LVLMs and offer helpful suggestions to mitigate the hallucination problem. Our training data and human annotation hallucination data will be made public soon.

翻译：大型视觉语言模型（LVLMs）近期取得了显著成功。然而，这些模型仍受限于幻觉问题，这限制了其在众多场景中的实用性。幻觉是指LVLM响应中不存在于视觉输入中的信息，这可能会带来重大后果的潜在风险。目前关于LVLM幻觉评估的研究工作有限。本文提出基于大语言模型的幻觉评估框架HaELM，该框架实现了与ChatGPT约95%的性能相当的效果，并具有低成本、可复现性、隐私保护和本地部署等额外优势。借助HaELM，我们对当前LVLM中的幻觉现象进行了评估。此外，我们分析了导致LVLM产生幻觉的因素，并为缓解幻觉问题提供了有益建议。我们的训练数据与人工标注的幻觉数据将很快公开。