Bayesian neural network posterior distributions have a great number of modes that correspond to the same network function. The abundance of such modes can make it difficult for approximate inference methods to do their job. Recent work has demonstrated the benefits of partial stochasticity for approximate inference in Bayesian neural networks; inference can be less costly and performance can sometimes be improved. I propose a structured way to select the deterministic subset of weights that removes neuron permutation symmetries, and therefore the corresponding redundant posterior modes. With a drastically simplified posterior distribution, the performance of existing approximate inference schemes is found to be greatly improved.

翻译：贝叶斯神经网络后验分布存在大量对应于同一网络函数的模态。此类模态的冗余性可能使近似推断方法难以有效工作。近期研究表明，部分随机性在贝叶斯神经网络的近似推断中具有优势：推断成本可能降低，性能有时也能得到提升。本文提出一种结构化方法，通过选择权重的确定性子集来消除神经元置换对称性，从而消除相应的冗余后验模态。在极大简化的后验分布条件下，现有近似推断方案的性能得到了显著提升。