Dynamic neural networks are a recent technique that promises a remedy for the increasing size of modern deep learning models by dynamically adapting their computational cost to the difficulty of the inputs. In this way, the model can adjust to a limited computational budget. However, the poor quality of uncertainty estimates in deep learning models makes it difficult to distinguish between hard and easy samples. To address this challenge, we present a computationally efficient approach for post-hoc uncertainty quantification in dynamic neural networks. We show that adequately quantifying and accounting for both aleatoric and epistemic uncertainty through a probabilistic treatment of the last layers improves the predictive performance and aids decision-making when determining the computational budget. In the experiments, we show improvements on CIFAR-100, ImageNet, and Caltech-256 in terms of accuracy, capturing uncertainty, and calibration error.

翻译：动态神经网络是一种最新技术，旨在通过根据输入难度动态调整计算成本，解决现代深度学习模型日益增大的问题。这样，模型可以适应有限的计算预算。然而，深度学习模型中不确定性估计的质量较差，使得难以区分困难样本和简单样本。为应对这一挑战，我们提出了一种计算高效的方法，用于动态神经网络中的事后不确定性量化。我们证明，通过对最后几层进行概率处理，充分量化并考虑偶然不确定性和认知不确定性，能够提升预测性能，并在确定计算预算时辅助决策。实验中，我们在CIFAR-100、ImageNet和Caltech-256数据集上展示了在准确性、不确定性捕捉和校准误差方面的改进。