Inference on modern Bayesian Neural Networks (BNNs) often relies on a variational inference treatment, imposing violated assumptions of independence and the form of the posterior. Traditional MCMC approaches avoid these assumptions at the cost of increased computation due to its incompatibility to subsampling of the likelihood. New Piecewise Deterministic Markov Process (PDMP) samplers permit subsampling, though introduce a model specific inhomogenous Poisson Process (IPPs) which is difficult to sample from. This work introduces a new generic and adaptive thinning scheme for sampling from these IPPs, and demonstrates how this approach can accelerate the application of PDMPs for inference in BNNs. Experimentation illustrates how inference with these methods is computationally feasible, can improve predictive accuracy, MCMC mixing performance, and provide informative uncertainty measurements when compared against other approximate inference schemes.
翻译:现代贝叶斯神经网络的推断通常依赖于变分推断方法,但这种方法会强加独立性假设及后验分布形式的错误假定。传统MCMC方法虽能避免这些假设,却因无法对似然函数进行子采样而需要增加计算成本。新型分段确定性马尔可夫过程(PDMP)采样器允许子采样,但其引入的模型特定非齐次泊松过程(IPPs)难以采样。本文提出了一种通用的自适应细化采样方案,可高效从这些IPPs中采样,并论证该方法如何加速PDMP在贝叶斯神经网络推断中的应用。实验表明,与其他近似推断方案相比,基于这些方法的推断在计算上可行,能提升预测精度、改进MCMC混合性能,并提供更具信息量的不确定性测量。