While powerful probabilistic models such as Gaussian Processes naturally have this property, deep neural networks often lack it. In this paper, we introduce Distance Aware Bottleneck (DAB), i.e., a new method for enriching deep neural networks with this property. Building on prior information bottleneck approaches, our method learns a codebook that stores a compressed representation of all inputs seen during training. The distance of a new example from this codebook can serve as an uncertainty estimate for the example. The resulting model is simple to train and provides deterministic uncertainty estimates by a single forward pass. Finally, our method achieves better out-of-distribution (OOD) detection and misclassification prediction than prior methods, including expensive ensemble methods, deep kernel Gaussian Processes, and approaches based on the standard information bottleneck.

翻译：尽管高斯过程等强大的概率模型天然具备这一特性，但深度神经网络往往缺乏该性质。本文提出距离感知瓶颈（DAB）方法——一种为深度神经网络赋予该特性的新方法。基于先验信息瓶颈方法，本方法通过学习码本存储训练期间所见全部输入的压缩表示。新样本与此码本的距离可作为该样本的不确定性估计。所得模型训练简单，仅需单次前向传播即可获得确定性不确定性估计。最终，本方法在分布外（OOD）检测和误分类预测方面优于现有方法，包括计算代价高昂的集成方法、深度核高斯过程以及基于标准信息瓶颈的方法。