Deep neural networks (DNNs) often produce overconfident out-of-distribution predictions, motivating Bayesian uncertainty quantification. The Linearized Laplace Approximation (LLA) achieves this by linearizing the DNN and applying Laplace inference to the resulting model. Importantly, the linear model is also used for prediction. We argue this linearization in the posterior may degrade fidelity to the true Laplace approximation. To alleviate this problem, without increasing significantly the computational cost, we propose the Quadratic Laplace Approximation (QLA). QLA approximates each second order factor in the approximate Laplace log-posterior using a rank-one factor obtained via efficient power iterations. QLA is expected to yield a posterior precision closer to that of the full Laplace without forming the full Hessian, which is typically intractable. For prediction, QLA also uses the linearized model. Empirically, QLA yields modest yet consistent uncertainty estimation improvements over LLA on five regression datasets.

翻译：深度神经网络（DNNs）常对分布外预测产生过度自信，这推动了贝叶斯不确定性量化的发展。线性化拉普拉斯近似（LLA）通过将DNN线性化并对所得模型应用拉普拉斯推断来实现这一目标。重要的是，线性模型也用于预测。我们认为，后验中的这种线性化可能会降低对真实拉普拉斯近似的保真度。为了缓解此问题，同时不显著增加计算成本，我们提出了二次拉普拉斯近似（QLA）。QLA通过高效的幂迭代获得秩一因子，用以近似近似拉普拉斯对数后验中的每个二阶因子。QLA有望在不形成通常难以处理的完整Hessian矩阵的情况下，产生更接近完整拉普拉斯近似的后验精度。对于预测，QLA同样使用线性化模型。实证表明，在五个回归数据集上，QLA相比LLA在不确定性估计方面取得了适度但一致的改进。