AI Uncertainty Quantification (UQ) has the potential to improve human decision-making beyond AI predictions alone by providing additional probabilistic information to users. The majority of past research on AI and human decision-making has concentrated on model explainability and interpretability, with little focus on understanding the potential impact of UQ on human decision-making. We evaluated the impact on human decision-making for instance-level UQ, calibrated using a strict scoring rule, in two online behavioral experiments. In the first experiment, our results showed that UQ was beneficial for decision-making performance compared to only AI predictions. In the second experiment, we found UQ had generalizable benefits for decision-making across a variety of representations for probabilistic information. These results indicate that implementing high quality, instance-level UQ for AI may improve decision-making with real systems compared to AI predictions alone.

翻译：AI不确定性量化（Uncertainty Quantification，UQ）除了提供AI预测结果外，还能为用户提供额外的概率信息，从而有望改善人类决策。以往关于AI与人类决策的研究大多集中于模型的可解释性和可解释性，而较少关注UQ对人类决策的潜在影响。我们通过两项在线行为实验，评估了基于严格评分规则校准的实例级UQ对人类决策的影响。在第一项实验中，结果表明，与仅依赖AI预测相比，UQ有助于提升决策表现。在第二项实验中，我们发现UQ在多种概率信息表示形式下均对决策具有可推广的益处。这些结果表明，与仅使用AI预测相比，为AI系统实施高质量的实例级UQ能够改善与实际系统交互时的决策效果。