Partial Bayesian neural networks (pBNNs) have been shown to perform competitively with fully Bayesian neural networks while only having a subset of the parameters be stochastic. Using sequential Monte Carlo (SMC) samplers as the inference method for pBNNs gives a non-parametric probabilistic estimation of the stochastic parameters, and has shown improved performance over parametric methods. In this paper we introduce a new SMC-based training method for pBNNs by utilising a guided proposal and incorporating gradient-based Markov kernels, which gives us better scalability on high dimensional problems. We show that our new method outperforms the state-of-the-art in terms of predictive performance and optimal loss. We also show that pBNNs scale well with larger batch sizes, resulting in significantly reduced training times and often better performance.

翻译：部分贝叶斯神经网络（pBNNs）已被证明能够与完全贝叶斯神经网络竞争，同时仅需对部分参数进行随机化处理。采用序列蒙特卡洛（SMC）采样器作为pBNNs的推断方法，可对随机参数进行非参数化的概率估计，并已显示出优于参数化方法的性能。本文通过引入引导式提议机制并结合基于梯度的马尔可夫核，提出了一种新的基于SMC的pBNNs训练方法，该方法在高维问题上具有更好的可扩展性。实验表明，新方法在预测性能与最优损失方面均优于现有最优方法。同时，我们验证了pBNNs能够适应更大的批量尺寸，从而显著缩短训练时间，并往往获得更优的性能表现。