As deep neural networks are more commonly deployed in high-stakes domains, their black-box nature makes uncertainty quantification challenging. We investigate the presentation of conformal prediction sets--a distribution-free class of methods for generating prediction sets with specified coverage--to express uncertainty in AI-advised decision-making. Through a large online experiment, we compare the utility of conformal prediction sets to displays of Top-1 and Top-k predictions for AI-advised image labeling. In a pre-registered analysis, we find that the utility of prediction sets for accuracy varies with the difficulty of the task: while they result in accuracy on par with or less than Top-1 and Top-k displays for easy images, prediction sets offer some advantage in assisting humans in labeling out-of-distribution (OOD) images in the setting that we studied, especially when the set size is small. Our results empirically pinpoint practical challenges of conformal prediction sets and provide implications on how to incorporate them for real-world decision-making.

翻译：随着深度神经网络在高风险领域中的广泛应用，其黑箱特性使不确定性量化变得具有挑战性。本研究探索了共形预测集——一种无需分布假设、可生成具有指定覆盖率的预测集的方法——在AI辅助决策中表达不确定性的呈现方式。通过大型在线实验，我们比较了共形预测集与Top-1及Top-k预测结果展示在AI辅助图像标注任务中的实用性。在预设分析中，我们发现预测集对准确率的效用随任务难度而变化：对于简单图像，其准确率与Top-1及Top-k展示持平或更低；而在我们研究的场景中，预测集在辅助人类标注分布外（OOD）图像时展现出一定优势，尤其在预测集规模较小时。实验结果实证性地指出了共形预测集的实际应用挑战，并为将其整合至真实世界决策提供了启示。