Dynamic neural networks are a recent technique that promises a remedy for the increasing size of modern deep learning models by dynamically adapting their computational cost to the difficulty of the input samples. In this way, the model can adjust to a limited computational budget. However, the poor quality of uncertainty estimates in deep learning models makes it difficult to distinguish between hard and easy samples. To address this challenge, we present a computationally efficient approach for post-hoc uncertainty quantification in dynamic neural networks. We show that adequately quantifying and accounting for both aleatoric and epistemic uncertainty through a probabilistic treatment of the last layers improves the predictive performance and aids decision-making when determining the computational budget. In the experiments, we show improvements on CIFAR-100 and ImageNet in terms of accuracy, capturing uncertainty, and calibration error.

翻译：动态神经网络是一种新兴技术，它通过根据输入样本的难度动态调整计算成本，有望缓解现代深度学习模型日益增长的规模问题。通过这种方式，模型能够适应有限的计算预算。然而，深度学习模型中不确定性估计质量较低，使得区分困难样本与简单样本变得困难。为应对这一挑战，本文提出了一种计算高效的动态神经网络事后不确定性量化方法。研究表明，通过对最后几层进行概率化处理，充分量化和考虑偶然不确定性与认知不确定性，能够提升预测性能，并在确定计算预算时辅助决策。在实验中，我们在CIFAR-100和ImageNet数据集上展示了准确率、不确定性捕捉能力以及校准误差方面的改进。