Although deep neural networks yield high classification accuracy given sufficient training data, their predictions are typically overconfident or under-confident, i.e., the prediction confidences cannot truly reflect the accuracy. Post-hoc calibration tackles this problem by calibrating the prediction confidences without re-training the classification model. However, current approaches assume congruence between test and validation data distributions, limiting their applicability to out-of-distribution scenarios. To this end, we propose a novel meta-set-based cascaded temperature regression method for post-hoc calibration. Our method tailors fine-grained scaling functions to distinct test sets by simulating various domain shifts through data augmentation on the validation set. We partition each meta-set into subgroups based on predicted category and confidence level, capturing diverse uncertainties. A regression network is then trained to derive category-specific and confidence-level-specific scaling, achieving calibration across meta-sets. Extensive experimental results on MNIST, CIFAR-10, and TinyImageNet demonstrate the effectiveness of the proposed method.

翻译：尽管深度神经网络在充足训练数据下能获得高分类准确率，但其预测通常存在过度自信或信心不足的问题，即预测置信度无法真实反映准确率。后验校准通过在不重新训练分类模型的情况下校准预测置信度来解决这一问题。然而，现有方法假设测试数据与验证数据分布一致，限制了其在分布外场景中的适用性。为此，我们提出一种基于元集的新型级联温度回归方法进行后验校准。该方法通过在验证集上进行数据增强模拟不同域偏移，为不同测试集定制细粒度缩放函数。我们根据预测类别和置信度水平将每个元集划分为子群体，捕获多样化的不确定性。随后训练回归网络推导类别特定与置信度水平特定的缩放系数，实现跨元集的校准。在MNIST、CIFAR-10和TinyImageNet上的大量实验结果表明了该方法的有效性。