Bayesian Neural Networks (BNNs) have become one of the promising approaches for uncertainty estimation due to the solid theorical foundations. However, the performance of BNNs is affected by the ability of catching uncertainty. Instead of only seeking the distribution of neural network weights by in-distribution (ID) data, in this paper, we propose a new Bayesian Neural Network with an Attached structure (ABNN) to catch more uncertainty from out-of-distribution (OOD) data. We first construct a mathematical description for the uncertainty of OOD data according to the prior distribution, and then develop an attached Bayesian structure to integrate the uncertainty of OOD data into the backbone network. ABNN is composed of an expectation module and several distribution modules. The expectation module is a backbone deep network which focuses on the original task, and the distribution modules are mini Bayesian structures which serve as attachments of the backbone. In particular, the distribution modules aim at extracting the uncertainty from both ID and OOD data. We further provide theoretical analysis for the convergence of ABNN, and experimentally validate its superiority by comparing with some state-of-the-art uncertainty estimation methods Code will be made available.

翻译：贝叶斯神经网络（BNN）凭借其坚实的理论基础，已成为不确定性估计领域最具前景的方法之一。然而，BNN的性能受限于其不确定性捕获能力。本文不再仅通过分布内（ID）数据求解神经网络权重的分布，而是提出一种具有附加结构的新型贝叶斯神经网络（ABNN），用于从分布外（OOD）数据中捕获更多不确定性。我们首先根据先验分布构建OOD数据不确定性的数学描述，然后开发一种附加贝叶斯结构，将OOD数据的不确定性集成至骨干网络中。ABNN由期望模块和多个分布模块组成：期望模块是专注于原始任务的骨干深度网络，而分布模块作为骨干网络的附加微型贝叶斯结构，旨在从ID和OOD数据中提取不确定性。我们进一步提供了ABNN收敛性的理论分析，并通过与多种先进不确定性估计方法的对比实验，验证了其优越性。代码将开源。