Traditional neural networks are simple to train but they produce overconfident predictions, while Bayesian neural networks provide good uncertainty quantification but optimizing them is time consuming. This paper introduces a new approach, direct uncertainty quantification (DirectUQ), that combines their advantages where the neural network directly models uncertainty in output space, and captures both aleatoric and epistemic uncertainty. DirectUQ can be derived as an alternative variational lower bound, and hence benefits from collapsed variational inference that provides improved regularizers. On the other hand, like non-probabilistic models, DirectUQ enjoys simple training and one can use Rademacher complexity to provide risk bounds for the model. Experiments show that DirectUQ and ensembles of DirectUQ provide a good tradeoff in terms of run time and uncertainty quantification, especially for out of distribution data.

翻译：传统神经网络易于训练，但会产生过于自信的预测，而贝叶斯神经网络虽然能提供良好的不确定性量化，但其优化过程耗时较长。本文提出一种新方法——直接不确定性量化（DirectUQ），该方法结合了二者的优势：神经网络直接对输出空间中的不确定性进行建模，并同时捕获偶然不确定性和认知不确定性。DirectUQ可作为变分下界的替代形式推导得出，因此受益于坍缩变分推断，从而提供改进的正则化项。另一方面，如同非概率模型，DirectUQ具有训练简单的特点，且可利用Rademacher复杂度为该模型提供风险界。实验表明，DirectUQ及其集成方法在运行时间和不确定性量化之间取得了良好平衡，尤其适用于分布外数据。