Bayesian neural network posterior distributions have a great number of modes that correspond to the same network function. The abundance of such modes can make it difficult for approximate inference methods to do their job. Recent work has demonstrated the benefits of partial stochasticity for approximate inference in Bayesian neural networks; inference can be less costly and performance can sometimes be improved. I propose a structured way to select the deterministic subset of weights that removes neuron permutation symmetries, and therefore the corresponding redundant posterior modes. With a drastically simplified posterior distribution, the performance of existing approximate inference schemes is found to be greatly improved.

翻译：贝叶斯神经网络后验分布存在大量对应于同一网络函数的多峰结构。此类模态的冗余性可能使近似推断方法难以有效工作。近期研究表明，部分随机性策略在贝叶斯神经网络近似推断中具有显著优势：既能降低推断成本，有时还能提升性能。本文提出一种结构化方法来确定权重的确定性子集，该方法能消除神经元排列对称性，从而消除相应的冗余后验模态。通过极大简化后验分布，现有近似推断方案的性能得到了显著提升。