While humans increasingly rely on large language models (LLMs), they are susceptible to generating inaccurate or false information, also known as "hallucinations". Technical advancements have been made in algorithms that detect hallucinated content by assessing the factuality of the model's responses and attributing sections of those responses to specific source documents. However, there is limited research on how to effectively communicate this information to users in ways that will help them appropriately calibrate their trust toward LLMs. To address this issue, we conducted a scenario-based study (N=104) to systematically compare the impact of various design strategies for communicating factuality and source attribution on participants' ratings of trust, preferences, and ease in validating response accuracy. Our findings reveal that participants preferred a design in which phrases within a response were color-coded based on the computed factuality scores. Additionally, participants increased their trust ratings when relevant sections of the source material were highlighted or responses were annotated with reference numbers corresponding to those sources, compared to when they received no annotation in the source material. Our study offers practical design guidelines to facilitate human-LLM collaboration and it promotes a new human role to carefully evaluate and take responsibility for their use of LLM outputs.

翻译：随着人类对大型语言模型（LLMs）的依赖日益加深，这些模型容易产生不准确或虚假信息，即所谓的"幻觉"。目前，通过评估模型响应的事实性并将响应内容归因于特定源文件来检测幻觉内容的算法已取得技术进展。然而，关于如何有效向用户传达此类信息以帮助其合理调整对LLMs信任度的研究仍较为有限。为解决这一问题，我们开展了一项基于场景的研究（N=104），系统比较了不同事实性与来源归因传达设计策略对参与者信任度评分、偏好及验证响应准确性的便捷性评价的影响。研究发现，参与者更青睐根据计算所得事实性评分对响应中的短语进行颜色编码的设计方案。此外，与未获得源材料标注的情况相比，当源材料相关部分被高亮显示或响应标注了对应来源的参考编号时，参与者的信任度评分显著提升。本研究为促进人类与LLM的协作提供了实用设计指南，并倡导人类应承担仔细评估并对LLM输出内容的使用负责的新角色。