Deep discriminative approaches like random forests and deep neural networks have recently found applications in many important real-world scenarios. However, deploying these learning algorithms in safety-critical applications raises concerns, particularly when it comes to ensuring confidence calibration for both in-distribution and out-of-distribution data points. Many popular methods for in-distribution (ID) calibration, such as isotonic regression and Platt's sigmoidal regression, exhibit excellent ID calibration performance. However, these methods are not calibrated for the entire feature space, leading to overconfidence in the case of out-of-distribution (OOD) samples. On the other end of the spectrum, existing out-of-distribution (OOD) calibration methods generally exhibit poor in-distribution (ID) calibration. In this paper, we address ID and OOD calibration problems jointly. We leveraged the fact that deep models, including both random forests and deep-nets, learn internal representations which are unions of polytopes with affine activation functions to conceptualize them both as partitioning rules of the feature space. We replace the affine function in each polytope populated by the training data with a Gaussian kernel. We propose sufficient conditions for our proposed methods to be consistent estimators of the corresponding class conditional densities. Moreover, our experiments on both tabular and vision benchmarks show that the proposed approaches obtain well-calibrated posteriors while mostly preserving or improving the classification accuracy of the original algorithm for in-distribution region, and extrapolates beyond the training data to handle out-of-distribution inputs appropriately.

翻译：深度判别方法（如随机森林和深度神经网络）近年来已在许多重要的实际场景中得到应用。然而，将这些学习算法部署到安全关键型应用中引发了担忧，尤其是在确保分布内和分布外数据点的置信度校准方面。许多流行的分布内（ID）校准方法，如等渗回归和Platt的S形回归，在ID校准上表现出色。但这些方法未针对整个特征空间进行校准，导致对分布外（OOD）样本过度自信。另一方面，现有的分布外校准方法通常分布内（ID）校准性能较差。本文联合解决了ID与OOD校准问题。我们利用深度模型（包括随机森林和深度网络）学习到的内部表示是带仿射激活函数的多面体并集这一事实，将二者均视为特征空间的划分规则。我们用高斯核替换训练数据所在每个多面体中的仿射函数。我们提出了保证所提方法能一致估计相应类条件密度的充分条件。此外，在表格数据和视觉基准上的实验表明，所提方法在保持或提升原始算法在分布内区域的分类精度的同时，获得了良好校准的后验概率，并能合理泛化至训练数据之外，恰当处理分布外输入。