Dynamic neural networks are a recent technique that promises a remedy for the increasing size of modern deep learning models by dynamically adapting their computational cost to the difficulty of the inputs. In this way, the model can adjust to a limited computational budget. However, the poor quality of uncertainty estimates in deep learning models makes it difficult to distinguish between hard and easy samples. To address this challenge, we present a computationally efficient approach for post-hoc uncertainty quantification in dynamic neural networks. We show that adequately quantifying and accounting for both aleatoric and epistemic uncertainty through a probabilistic treatment of the last layers improves the predictive performance and aids decision-making when determining the computational budget. In the experiments, we show improvements on CIFAR-100, ImageNet, and Caltech-256 in terms of accuracy, capturing uncertainty, and calibration error.

翻译：动态神经网络是一种近期技术，旨在通过根据输入难度动态调整计算成本，来缓解现代深度学习模型规模不断增长的问题。通过这种方式，模型能够适应有限的计算预算。然而，深度学习模型中不确定性估计质量较差，导致难以区分困难样本与简单样本。为应对这一挑战，我们提出了一种计算高效的方法，用于动态神经网络中的事后不确定性量化。研究表明，通过对最后几层进行概率化处理，充分量化并考虑偶然不确定性与认知不确定性，能够提升预测性能，并在确定计算预算时辅助决策。实验结果表明，在CIFAR-100、ImageNet和Caltech-256数据集上，该方法在准确率、不确定性捕捉及校准误差方面均有改进。