The widespread adoption and transformative effects of large language models (LLMs) have sparked concerns regarding their capacity to produce inaccurate and fictitious content, referred to as `hallucinations'. Given the potential risks associated with hallucinations, humans should be able to identify them. This research aims to understand the human perception of LLM hallucinations by systematically varying the degree of hallucination (genuine, minor hallucination, major hallucination) and examining its interaction with warning (i.e., a warning of potential inaccuracies: absent vs. present). Participants (N=419) from Prolific rated the perceived accuracy and engaged with content (e.g., like, dislike, share) in a Q/A format. Results indicate that humans rank content as truthful in the order genuine > minor hallucination > major hallucination and user engagement behaviors mirror this pattern. More importantly, we observed that warning improves hallucination detection without significantly affecting the perceived truthfulness of genuine content. We conclude by offering insights for future tools to aid human detection of hallucinations.

翻译：摘要：大型语言模型（LLMs）的广泛采用及其变革性影响引发了对其产生不准确和虚构内容（即“幻觉”）能力的担忧。鉴于幻觉的潜在风险，人类应能识别此类内容。本研究旨在通过系统改变幻觉程度（真实、轻微幻觉、严重幻觉）并探讨其与警告（即对潜在不准确性的警告：无警告 vs. 有警告）的交互作用，来理解人类对LLM幻觉的感知。来自Prolific的419名参与者在问答格式下评估了感知准确性，并对内容进行互动（如喜欢、不喜欢、分享）。结果表明，人类对内容真实性的排序为：真实内容 > 轻微幻觉 > 严重幻觉，用户互动行为也呈现相同模式。更重要的是，我们观察到警告能提升幻觉检测能力，同时不影响对真实内容准确性的感知。最后，我们为未来辅助人类检测幻觉的工具提供了见解。