As deep neural networks are more commonly deployed in high-stakes domains, their lack of interpretability makes uncertainty quantification challenging. We investigate the effects of presenting conformal prediction sets$\unicode{x2013}$a method for generating valid confidence sets in distribution-free uncertainty quantification$\unicode{x2013}$to express uncertainty in AI-advised decision-making. Through a large pre-registered experiment, we compare the utility of conformal prediction sets to displays of Top-1 and Top-k predictions for AI-advised image labeling. We find that the utility of prediction sets for accuracy varies with the difficulty of the task: while they result in accuracy on par with or less than Top-1 and Top-k displays for easy images, prediction sets excel at assisting humans in labeling out-of-distribution (OOD) images especially when the set size is small. Our results empirically pinpoint the practical challenges of conformal prediction sets and provide implications on how to incorporate them for real-world decision-making.

翻译：随着深度神经网络在高风险领域的广泛应用，其缺乏可解释性使得不确定性量化面临挑战。本文研究了一种用于无分布假设下生成有效置信集的共形预测集方法，探讨将其用于表达AI辅助决策中的不确定性所产生的效果。通过一项大规模预注册实验，我们比较了共形预测集与Top-1和Top-k预测展示在AI辅助图像标注任务中的效用。研究发现，预测集对准确性的效用随任务难度而变化：对于简单图像，其准确性与Top-1和Top-k展示相当或更低；但在标注分布外图像时，尤其当预测集规模较小时，预测集能有效辅助人类提升标注效果。本实验结果实证揭示了共形预测集在实际应用中的挑战，并为将其融入真实世界决策提供了启示。