Dynamic neural networks are a recent technique that promises a remedy for the increasing size of modern deep learning models by dynamically adapting their computational cost to the difficulty of the input samples. In this way, the model can adjust to a limited computational budget. However, the poor quality of uncertainty estimates in deep learning models makes it difficult to distinguish between hard and easy samples. To address this challenge, we present a computationally efficient approach for post-hoc uncertainty quantification in dynamic neural networks. We show that adequately quantifying and accounting for both aleatoric and epistemic uncertainty through a probabilistic treatment of the last layers improves the predictive performance and aids decision-making when determining the computational budget. In the experiments, we show improvements on CIFAR-100 and ImageNet in terms of accuracy, capturing uncertainty, and calibration error.

翻译：动态神经网络是一种新兴技术，可通过动态调整计算成本以适应输入样本的难度，从而缓解现代深度学习模型日益增长的计算负担。这种机制使模型能够适应有限的计算预算。然而，深度学习模型中不确定性估计质量低下，导致难以区分困难样本与简单样本。针对这一挑战，我们提出了一种计算高效的方法，用于对动态神经网络进行事后不确定性量化。研究表明，通过对网络最后几层进行概率化处理，充分量化并考虑偶然不确定性与认知不确定性，可以提升预测性能，并在确定计算预算时辅助决策。在实验中，我们在CIFAR-100和ImageNet数据集上展示了该方法在准确率、不确定性捕捉以及校准误差方面的改进。