We introduce a scalable method to approximate the kernel of the Linearized Laplace Approximation (LLA). For this, we use a surrogate deep neural network (DNN) that learns a compact feature representation whose inner product replicates the Neural Tangent Kernel (NTK). This avoids the need to compute large Jacobians. Training relies solely on efficient Jacobian-vector products, allowing to compute predictive uncertainty on large-scale pre-trained DNNs. Experimental results show similar or improved uncertainty estimation and calibration compared to existing LLA approximations. Notwithstanding, biasing the learned kernel significantly enhances out-of-distribution detection. This remarks the benefits of the proposed method for finding better kernels than the NTK in the context of LLA to compute prediction uncertainty given a pre-trained DNN.

翻译：本文提出了一种可扩展的方法来近似线性化拉普拉斯近似（LLA）的核。为此，我们使用一个代理深度神经网络（DNN），该网络学习一个紧凑的特征表示，其内积可复现神经正切核（NTK）。这避免了计算大型雅可比矩阵的需求。训练仅依赖于高效的雅可比-向量乘积，从而能够在大型预训练DNN上计算预测不确定性。实验结果表明，与现有的LLA近似方法相比，本文方法在不确定性估计和校准方面表现相当或更优。此外，对学习到的核进行偏置可显著提升分布外检测性能。这凸显了所提方法在LLA背景下，针对给定预训练DNN计算预测不确定性时，能够找到优于NTK的核的益处。