Reliable uncertainty estimates are crucial for deploying pretrained models; yet, many strong methods for quantifying uncertainty require retraining, Monte Carlo sampling, or expensive second-order computations and may alter a frozen backbone's predictions. To address this, we introduce Gaussian Process Activations (GAPA), a post-hoc method that shifts Bayesian modeling from weights to activations. GAPA replaces standard nonlinearities with Gaussian-process activations whose posterior mean exactly matches the original activation, preserving the backbone's point predictions by construction while providing closed-form epistemic variances in activation space. To scale to modern architectures, we use a sparse variational inducing-point approximation over cached training activations, combined with local k-nearest-neighbor subset conditioning, enabling deterministic single-pass uncertainty propagation without sampling, backpropagation, or second-order information. Across regression, classification, image segmentation, and language modeling, GAPA matches or outperforms strong post-hoc baselines in calibration and out-of-distribution detection while remaining efficient at test time.

翻译：可靠的**不确定性估计**对于部署预训练模型至关重要；然而，许多量化不确定性的有效方法需要重新训练、蒙特卡洛采样或昂贵的二阶计算，并且可能改变冻结主干网络的预测。为解决此问题，我们引入了**高斯过程激活**（Gaussian Process Activations，GAPA），一种将贝叶斯建模从权重转移到激活的事后方法。GAPA用高斯过程激活替换标准非线性激活函数，其后验均值与原始激活完全匹配，从而在结构上保持主干网络的点预测，同时在激活空间中提供封闭形式的认知方差。为适应现代架构，我们基于缓存的训练激活使用稀疏变分诱导点近似，并结合局部k近邻子集条件化，实现了无需采样、反向传播或二阶信息的确定性单次前向不确定性传播。在回归、分类、图像分割和语言建模任务中，GAPA在校准和分布外检测方面达到或优于强事后基线方法，同时在测试时保持高效。