Bayesian deep learning and conformal prediction are two methods that have been used to convey uncertainty and increase safety in machine learning systems. We focus on combining Bayesian deep learning with split conformal prediction and how this combination effects out-of-distribution coverage; particularly in the case of multiclass image classification. We suggest that if the model is generally underconfident on the calibration set, then the resultant conformal sets may exhibit worse out-of-distribution coverage compared to simple predictive credible sets. Conversely, if the model is overconfident on the calibration set, the use of conformal prediction may improve out-of-distribution coverage. We evaluate prediction sets as a result of combining split conformal methods and neural networks trained with (i) stochastic gradient descent, (ii) deep ensembles, and (iii) mean-field variational inference. Our results suggest that combining Bayesian deep learning models with split conformal prediction can, in some cases, cause unintended consequences such as reducing out-of-distribution coverage.

翻译：贝叶斯深度学习和共形预测是两种用于在机器学习系统中传达不确定性并提升安全性的方法。我们聚焦于将贝叶斯深度学习与分割共形预测相结合，并探讨这种结合对分布外覆盖的影响，特别是在多类别图像分类场景中。我们认为，若模型在校准集上普遍倾向于欠自信（即预测置信度偏低），则由此生成的共形集与简单预测可信集相比，可能表现出更差的分布外覆盖性。反之，若模型在校准集上过度自信，使用共形预测则可改善分布外覆盖。我们评估了通过结合分割共形方法与以下三种方式训练的神经网络所得预测集：(i) 随机梯度下降法、(ii) 深度集成法、(iii) 平均场变分推断法。研究结果表明，在某些情况下，将贝叶斯深度学习模型与分割共形预测结合可能引发意外后果，例如降低分布外覆盖性能。