Convolutional image classifiers can achieve high predictive accuracy, but quantifying their uncertainty remains an unresolved challenge, hindering their deployment in consequential settings. Existing uncertainty quantification techniques, such as Platt scaling, attempt to calibrate the network's probability estimates, but they do not have formal guarantees. We present an algorithm that modifies any classifier to output a predictive set containing the true label with a user-specified probability, such as 90%. The algorithm is simple and fast like Platt scaling, but provides a formal finite-sample coverage guarantee for every model and dataset. Our method modifies an existing conformal prediction algorithm to give more stable predictive sets by regularizing the small scores of unlikely classes after Platt scaling. In experiments on both Imagenet and Imagenet-V2 with ResNet-152 and other classifiers, our scheme outperforms existing approaches, achieving coverage with sets that are often factors of 5 to 10 smaller than a stand-alone Platt scaling baseline.

翻译：革命图像分类器可以实现高预测准确度, 但量化其不确定性仍然是一个尚未解决的挑战, 阻碍在相应环境下的部署 。 现有的不确定性量化技术, 如 Platt 缩放, 试图校准网络的概率估计, 但是它们没有正式的保证 。 我们提出了一个算法, 修改任何分类器来输出包含真实标签的预测集, 带有用户指定概率, 如 90% 。 算法简单而快速, 像 Platt 缩放一样, 但为每个模型和数据集提供一个正式的有限抽样覆盖保障 。 我们的方法修改现有的符合性预测算法, 以便通过在 Platt 缩放后对不常见的分类小分数进行常规化来提供更稳定的预测 。 在与 ResNet-152 和其他分类器进行的图像网 V2 实验中, 我们的算法超越了现有方法, 其覆盖范围往往比独立普拉特 缩放基准小5到 10 。