Although much research has focused on AI explanations to support decisions in complex information-seeking tasks such as fact-checking, the role of evidence is surprisingly under-researched. In our study, we systematically varied explanation type, AI prediction certainty, and correctness of AI system advice for non-expert participants, who evaluated the veracity of claims and AI system predictions. Participants were provided the option of easily inspecting the underlying evidence. We found that participants consistently relied on evidence to validate AI claims across all experimental conditions. When participants were presented with natural language explanations, evidence was used less frequently although they relied on it when these explanations seemed insufficient or flawed. Qualitative data suggests that participants attempted to infer evidence source reliability, despite source identities being deliberately omitted. Our results demonstrate that evidence is a key ingredient in how people evaluate the reliability of information presented by an AI system and, in combination with natural language explanations, offers valuable support for decision-making. Further research is urgently needed to understand how evidence ought to be presented and how people engage with it in practice.

翻译：尽管大量研究聚焦于AI解释如何支持复杂信息检索任务（如事实核查）中的决策，但证据的作用却令人惊讶地缺乏深入研究。在我们的研究中，我们系统性地操纵了解释类型、AI预测确定性以及AI系统建议的正确性，让非专业参与者评估声明的真实性及AI系统预测。参与者可选择便捷地查看底层证据。我们发现，在所有实验条件下，参与者始终依赖证据来验证AI声明。当参与者获得自然语言解释时，证据使用频率降低，但当这些解释显得不足或有缺陷时，他们仍会依赖证据。定性数据表明，尽管来源身份被刻意隐藏，参与者仍试图推断证据来源的可靠性。我们的研究结果表明，证据是人们评估AI系统所呈现信息可靠性的关键要素，并且与自然语言解释相结合，能为决策提供有价值的支持。亟需进一步研究以理解证据应如何呈现以及人们在实践中如何与之互动。