Modern deep learning models remain notoriously prone to overconfidence, limiting their reliability in high-stakes applications. Bayesian methods aim to counter this by learning a distribution over model parameters, and recent advances now make this feasible for large-scale architectures at costs comparable to AdamW. However, a challenge remains at test time: predictions must be averaged across many forward passes with weights sampled from the posterior, which is prohibitively expensive. Variance propagation offers an efficient alternative, computing layer-wise analytical approximations of uncertainty in a single forward pass. While such techniques are effective for MLPs, their extension to modern architectures remains challenging, due to increased depth and diversity of layer types. To fill this gap, we propose Calibrated Variance Propagation (CVP), which introduces a new propagation method for normalization layers, combines it with recent techniques for handling activation functions, and absorbs residual error through a light calibration step. CVP yields comparably accurate uncertainty estimates to MC sampling across transformers and CNNs, at a fraction of the cost. Against prior variance propagation work, CVP improves coverage at $0.5\%$ risk from $8.2\%$ to $14.6\%$ with BEiT-3 on Visual Reasoning (NLVR2) and from $2.6\%$ to $10.8\%$ with ViLT on VQAv2, with gains extending to convolutional architectures.

翻译：现代深度学习模型仍以过度自信著称，这限制了其在高风险应用中的可靠性。贝叶斯方法通过学习模型参数上的分布来应对这一问题，近期进展已使其在代价与AdamW相当的前提下适用于大规模架构。然而，测试阶段仍存挑战：预测需要对从后验中采样的权重进行多次前向传播的平均，这代价高昂。方差传播提供了一种高效替代方案，通过单次前向传播计算逐层解析近似的不确定性。尽管此类方法对MLP有效，但因其深度增加和层类型多样性，拓展至现代架构仍具挑战。为填补这一空白，我们提出校准方差传播(CVP)，该方法引入了针对归一化层的新传播机制，结合了处理激活函数的最新方法，并通过轻量校准步骤吸收残差误差。CVP能在Transformers和CNN上以极低代价获得与蒙特卡洛采样精度相当的不确定性估计。相较于先前的方差传播工作，CVP在基于BEiT-3的视觉推理(NLVR2)任务中将$0.5\%$风险下的覆盖率从$8.2\%$提升至$14.6\%$，在基于ViLT的VQAv2任务中从$2.6\%$提升至$10.8\%$，且增益延伸至卷积架构。