Traditional neural networks are simple to train but they produce overconfident predictions, while Bayesian neural networks provide good uncertainty quantification but optimizing them is time consuming. This paper introduces a new approach, direct uncertainty quantification (DirectUQ), that combines their advantages where the neural network directly models uncertainty in output space, and captures both aleatoric and epistemic uncertainty. DirectUQ can be derived as an alternative variational lower bound, and hence benefits from collapsed variational inference that provides improved regularizers. On the other hand, like non-probabilistic models, DirectUQ enjoys simple training and one can use Rademacher complexity to provide risk bounds for the model. Experiments show that DirectUQ and ensembles of DirectUQ provide a good tradeoff in terms of run time and uncertainty quantification, especially for out of distribution data.

翻译：传统神经网络易于训练，但会产生过度自信的预测；而贝叶斯神经网络虽能提供良好的不确定性量化，但其优化过程耗时。本文提出一种新方法——直接不确定性量化（DirectUQ），该方法综合了二者的优势：神经网络直接在输出空间中对不确定性建模，并同时捕获偶然不确定性和认知不确定性。DirectUQ可推导为一种替代变分下界，从而受益于能提供改进正则化器的坍缩变分推理。另一方面，与非概率模型类似，DirectUQ训练简单，且可利用Rademacher复杂度为模型提供风险边界。实验表明，DirectUQ及其集成方法在运行时间与不确定性量化之间实现了良好权衡，尤其适用于分布外数据。