Bayesian model reduction provides an efficient approach for comparing the performance of all nested sub-models of a model, without re-evaluating any of these sub-models. Until now, Bayesian model reduction has been applied mainly in the computational neuroscience community on simple models. In this paper, we formulate and apply Bayesian model reduction to perform principled pruning of Bayesian neural networks, based on variational free energy minimization. Direct application of Bayesian model reduction, however, gives rise to approximation errors. Therefore, a novel iterative pruning algorithm is presented to alleviate the problems arising with naive Bayesian model reduction, as supported experimentally on the publicly available UCI datasets for different inference algorithms. This novel parameter pruning scheme solves the shortcomings of current state-of-the-art pruning methods that are used by the signal processing community. The proposed approach has a clear stopping criterion and minimizes the same objective that is used during training. Next to these benefits, our experiments indicate better model performance in comparison to state-of-the-art pruning schemes.

翻译：贝叶斯模型约简提供了一种高效方法，可在不重新评估任何子模型的情况下比较模型所有嵌套子模型的性能。迄今为止，贝叶斯模型约简主要应用于计算神经科学领域的简单模型。本文基于变分自由能最小化，将贝叶斯模型约简方法进行公式化并应用于贝叶斯神经网络的原则性剪枝。然而，直接应用贝叶斯模型约简会导致近似误差。为此，我们提出一种新颖的迭代剪枝算法，以缓解朴素贝叶斯模型约简带来的问题，并在公开UCI数据集上通过不同推理算法进行了实验验证。这种新颖的参数剪枝方案解决了当前信号处理领域最先进剪枝方法的缺陷。所提出的方法具有明确的停止准则，且最小化与训练阶段相同的目标函数。除了这些优势外，我们的实验表明，与现有最先进剪枝方案相比，该模型性能更优。