Uncertainty estimation aims to evaluate the confidence of a trained deep neural network. However, existing uncertainty estimation approaches rely on low-dimensional distributional assumptions and thus suffer from the high dimensionality of latent features. Existing approaches tend to focus on uncertainty on discrete classification probabilities, which leads to poor generalizability to uncertainty estimation for other tasks. Moreover, most of the literature requires seeing the out-of-distribution (OOD) data in the training for better estimation of uncertainty, which limits the uncertainty estimation performance in practice because the OOD data are typically unseen. To overcome these limitations, we propose a new framework using data-adaptive high-dimensional hypothesis testing for uncertainty estimation, which leverages the statistical properties of the feature representations. Our method directly operates on latent representations and thus does not require retraining the feature encoder under a modified objective. The test statistic relaxes the feature distribution assumptions to high dimensionality, and it is more discriminative to uncertainties in the latent representations. We demonstrate that encoding features with Bayesian neural networks can enhance testing performance and lead to more accurate uncertainty estimation. We further introduce a family-wise testing procedure to determine the optimal threshold of OOD detection, which minimizes the false discovery rate (FDR). Extensive experiments validate the satisfactory performance of our framework on uncertainty estimation and task-specific prediction over a variety of competitors. The experiments on the OOD detection task also show satisfactory performance of our method when the OOD data are unseen in the training. Codes are available at https://github.com/HKU-MedAI/bnn_uncertainty.

翻译：不确定性估计旨在评估已训练深度神经网络的置信度。然而，现有不确定性估计方法依赖低维分布假设，因而难以处理隐特征的高维特性。现有方法往往聚焦于离散分类概率的不确定性，导致其难以泛化至其他任务的不确定性估计。此外，多数文献要求在训练时观测到分布外（OOD）数据以提升不确定性估计效果，这在实际应用中限制了估计性能——因为OOD数据通常不可见。为克服这些局限，我们提出一种新框架，通过数据自适应的高维假设检验实现不确定性估计，该方法利用特征表示的统计特性。我们的方法直接作用于隐层表示，因此无需在修改后的目标下重新训练特征编码器。检验统计量将特征分布假设松弛至高维情形，对隐层表示中的不确定性具有更强判别力。我们证明，采用贝叶斯神经网络编码特征可提升检验性能，从而获得更精确的不确定性估计。进一步，我们引入族系检验流程来确定OOD检测的最优阈值，以最小化错误发现率（FDR）。大量实验验证了我们的框架在不确定性估计和任务特定预测上优于多种竞争方法。针对OOD检测任务的实验表明，当OOD数据在训练中不可见时，我们的方法仍表现优异。代码见https://github.com/HKU-MedAI/bnn_uncertainty。