Bayesian inference for neural networks, or Bayesian deep learning, has the potential to provide well-calibrated predictions with quantified uncertainty and robustness. However, the main hurdle for Bayesian deep learning is its computational complexity due to the high dimensionality of the parameter space. In this work, we propose a novel scheme that addresses this limitation by constructing a low-dimensional subspace of the neural network parameters-referred to as an active subspace-by identifying the parameter directions that have the most significant influence on the output of the neural network. We demonstrate that the significantly reduced active subspace enables effective and scalable Bayesian inference via either Monte Carlo (MC) sampling methods, otherwise computationally intractable, or variational inference. Empirically, our approach provides reliable predictions with robust uncertainty estimates for various regression tasks.

翻译：贝叶斯推理用于神经网络（即贝叶斯深度学习）具有提供校准良好且带有量化不确定性与鲁棒性的预测的潜力。然而，贝叶斯深度学习的主要障碍在于参数空间的高维性所导致的计算复杂性。本文提出了一种新方案，通过识别对神经网络输出影响最显著的参数方向，构建神经网络参数的低维子空间（称为主动子空间），从而解决这一局限性。我们证明，显著降维后的主动子空间能够通过蒙特卡洛采样方法（否则在计算上不可行）或变分推理实现有效且可扩展的贝叶斯推断。实验表明，我们的方法为各类回归任务提供了具有稳健不确定性估计的可靠预测。