Deep neural network ensembles are powerful tools for uncertainty quantification, which have recently been re-interpreted from a Bayesian perspective. However, current methods inadequately leverage second-order information of the loss landscape, despite the recent availability of efficient Hessian approximations. We propose a novel approximate Bayesian inference method that modifies deep ensembles to incorporate Stein Variational Newton updates. Our approach uniquely integrates scalable modern Hessian approximations, achieving faster convergence and more accurate posterior distribution approximations. We validate the effectiveness of our method on diverse regression and classification tasks, demonstrating superior performance with a significantly reduced number of training epochs compared to existing ensemble-based methods, while enhancing uncertainty quantification and robustness against overfitting.

翻译：深度神经网络集成是用于不确定性量化的强大工具，最近已从贝叶斯视角被重新阐释。然而，尽管目前已有高效的Hessian近似方法可用，现有技术未能充分利用损失函数的二阶信息。我们提出了一种新颖的近似贝叶斯推断方法，通过引入斯坦变分牛顿更新来改进深度集成。该方法创新性地整合了可扩展的现代Hessian近似技术，实现了更快的收敛速度和更精确的后验分布逼近。我们在多种回归与分类任务上验证了该方法的有效性，结果表明：相较于现有基于集成的方法，本方法在显著减少训练轮数的同时，展现出更优的性能表现，并增强了不确定性量化能力与抗过拟合鲁棒性。