Bayesian approaches for learning deep neural networks (BNN) have been received much attention and successfully applied to various applications. Particularly, BNNs have the merit of having better generalization ability as well as better uncertainty quantification. For the success of BNN, search an appropriate architecture of the neural networks is an important task, and various algorithms to find good sparse neural networks have been proposed. In this paper, we propose a new node-sparse BNN model which has good theoretical properties and is computationally feasible. We prove that the posterior concentration rate to the true model is near minimax optimal and adaptive to the smoothness of the true model. In particular the adaptiveness is the first of its kind for node-sparse BNNs. In addition, we develop a novel MCMC algorithm which makes the Bayesian inference of the node-sparse BNN model feasible in practice.

翻译：贝叶斯方法用于深度神经网络（BNN）的学习已受到广泛关注，并成功应用于各种任务。特别是，BNN具有更好的泛化能力和更优的不确定性量化优势。为成功构建BNN，搜索合适的神经网络架构是一项重要任务，目前已提出多种算法来寻找良好的稀疏神经网络。本文提出了一种新的节点稀疏BNN模型，该模型具有良好的理论性质且计算可行。我们证明，该模型的后验收缩率关于真实模型达到近极小最优，并能自适应于真实模型的平滑性。特别地，这种自适应性在节点稀疏BNN中尚属首次。此外，我们开发了一种新颖的MCMC算法，使得节点稀疏BNN模型的贝叶斯推断在实践中可行。