Large Vision-Language Models (LVLMs) have recently achieved remarkable success. However, LVLMs are still plagued by the hallucination problem, which limits the practicality in many scenarios. Hallucination refers to the information of LVLMs' responses that does not exist in the visual input, which poses potential risks of substantial consequences. There has been limited work studying hallucination evaluation in LVLMs. In this paper, we propose Hallucination Evaluation based on Large Language Models (HaELM), an LLM-based hallucination evaluation framework. HaELM achieves an approximate 95% performance comparable to ChatGPT and has additional advantages including low cost, reproducibility, privacy preservation and local deployment. Leveraging the HaELM, we evaluate the hallucination in current LVLMs. Furthermore, we analyze the factors contributing to hallucination in LVLMs and offer helpful suggestions to mitigate the hallucination problem. Our training data and human annotation hallucination data will be made public soon.

翻译：近年来，大规模视觉-语言模型(LVLMs)取得了显著进展。然而，LVLMs仍受到幻觉问题的困扰，这限制了它们在许多场景中的实用性。幻觉是指LVLMs响应中不包含于视觉输入中的信息，这可能带来重大风险的潜在隐患。目前针对LVLMs的幻觉评估研究较少。本文提出基于大语言模型的幻觉评估框架——HaELM，这是一种基于LLM的幻觉评估方法。HaELM实现了与ChatGPT约95%性能近似的结果，并具有成本低廉、可复现、保护隐私及支持本地部署等额外优势。通过HaELM，我们对当前LVLMs中的幻觉现象进行评估。此外，我们分析了导致LVLMs幻觉的成因，并提出了缓解幻觉问题的建议。我们的训练数据和人工标注的幻觉数据将很快公开。