As deep neural networks are more commonly deployed in high-stakes domains, their lack of interpretability makes uncertainty quantification challenging. We investigate the effects of presenting conformal prediction sets$\unicode{x2013}$a method for generating valid confidence sets in distribution-free uncertainty quantification$\unicode{x2013}$to express uncertainty in AI-advised decision-making. Through a large online experiment, we compare the utility of conformal prediction sets to displays of Top-$1$ and Top-$k$ predictions for AI-advised image labeling. In a pre-registered analysis, we find that the utility of prediction sets for accuracy varies with the difficulty of the task: while they result in accuracy on par with or less than Top-$1$ and Top-$k$ displays for easy images, prediction sets excel at assisting humans in labeling out-of-distribution (OOD) images, especially when the set size is small. Our results empirically pinpoint the practical challenges of conformal prediction sets and provide implications on how to incorporate them for real-world decision-making.

翻译：随着深度神经网络在高风险领域中的部署日益普遍，其缺乏可解释性使得不确定性量化成为挑战。我们研究了展示共形预测集（一种在无分布假设下生成有效置信集的不确定性量化方法）对AI辅助决策中不确定性表达的影响。通过一项大规模在线实验，我们将共形预测集的实用性与AI辅助图像标注中Top-1和Top-k预测展示进行了比较。在预先注册的分析中，我们发现预测集在准确性上的实用性随任务难度而变化：对于简单图片，其准确性与Top-1和Top-k展示持平或略低；但在标注分布外（OOD）图片时，预测集尤其在集合规模较小时能显著提升辅助效果。我们的结果实证性地指出了共形预测集的实际挑战，并为其融入现实世界决策提供了启示。