In many real world contexts, successful human-AI collaboration requires humans to productively integrate complementary sources of information into AI-informed decisions. However, in practice human decision-makers often lack understanding of what information an AI model has access to in relation to themselves. There are few available guidelines regarding how to effectively communicate about unobservables: features that may influence the outcome, but which are unavailable to the model. In this work, we conducted an online experiment to understand whether and how explicitly communicating potentially relevant unobservables influences how people integrate model outputs and unobservables when making predictions. Our findings indicate that presenting prompts about unobservables can change how humans integrate model outputs and unobservables, but do not necessarily lead to improved performance. Furthermore, the impacts of these prompts can vary depending on decision-makers' prior domain expertise. We conclude by discussing implications for future research and design of AI-based decision support tools.
翻译:在许多现实场景中,成功的人机协作要求人类能够将互补性的信息源有效整合到基于人工智能的决策中。然而在实践中,人类决策者往往缺乏对人工智能模型相对于自身所能获取的信息范围的理解。当前关于如何有效沟通"不可观测要素"(即可能影响结果但模型无法获取的特征)的指导方针十分有限。本研究通过在线实验探究:明确传达潜在相关的不可观测要素是否以及如何影响人们在预测时整合模型输出与不可观测要素的方式。实验结果表明,呈现关于不可观测要素的提示可以改变人类整合模型输出与不可观测要素的方式,但未必能提升决策表现。此外,此类提示的效果会因决策者的先验领域专业知识而异。最后,我们讨论了相关发现对未来基于人工智能的决策支持工具研究与设计的启示。