Traditional neural networks are simple to train but they produce overconfident predictions, while Bayesian neural networks provide good uncertainty quantification but optimizing them is time consuming. This paper introduces a new approach, direct uncertainty quantification (DirectUQ), that combines their advantages where the neural network directly outputs the mean and variance of the last layer. DirectUQ can be derived as an alternative variational lower bound, and hence benefits from collapsed variational inference that provides improved regularizers. On the other hand, like non-probabilistic models, DirectUQ enjoys simple training and one can use Rademacher complexity to provide risk bounds for the model. Experiments show that DirectUQ and ensembles of DirectUQ provide a good tradeoff in terms of run time and uncertainty quantification, especially for out of distribution data.

翻译：传统神经网络易于训练但会产生过度自信的预测，而贝叶斯神经网络虽然能提供良好的不确定性量化，但其优化过程耗时较长。本文提出了一种新方法——直接不确定性量化（DirectUQ），该方法结合了两者的优势，使神经网络直接输出最后一层的均值和方差。DirectUQ可推导为一种替代变分下界，从而受益于能提供改进正则化项的折叠变分推理。另一方面，与非概率模型类似，DirectUQ具有训练简单的特点，且可利用Rademacher复杂度为该模型提供风险边界。实验表明，DirectUQ及其集成方法在运行时间和不确定性量化方面实现了良好平衡，尤其适用于分布外数据。