Uncertainty quantification is central to safe and efficient deployments of deep learning models, yet many computationally practical methods lack lacking rigorous theoretical motivation. Random network distillation (RND) is a lightweight technique that measures novelty via prediction errors against a fixed random target. While empirically effective, it has remained unclear what uncertainties RND measures and how its estimates relate to other approaches, e.g. Bayesian inference or deep ensembles. This paper establishes these missing theoretical connections by analyzing RND within the neural tangent kernel framework in the limit of infinite network width. Our analysis reveals two central findings in this limit: (1) The uncertainty signal from RND -- its squared self-predictive error -- is equivalent to the predictive variance of a deep ensemble. (2) By constructing a specific RND target function, we show that the RND error distribution can be made to mirror the centered posterior predictive distribution of Bayesian inference with wide neural networks. Based on this equivalence, we moreover devise a posterior sampling algorithm that generates i.i.d. samples from an exact Bayesian posterior predictive distribution using this modified \textit{Bayesian RND} model. Collectively, our findings provide a unified theoretical perspective that places RND within the principled frameworks of deep ensembles and Bayesian inference, and offer new avenues for efficient yet theoretically grounded uncertainty quantification methods.

翻译：不确定性量化是深度学习模型安全高效部署的核心，然而许多计算上可行的方法缺乏严格的理论依据。随机网络蒸馏是一种轻量级技术，通过预测误差相对于固定随机目标来衡量新颖性。尽管经验上有效，但RND究竟测量何种不确定性，其估计值与其他方法（如贝叶斯推断或深度集成）的关系如何，始终不甚明确。本文通过在无限网络宽度极限下的神经正切核框架中分析RND，建立了这些缺失的理论联系。我们的分析揭示了该极限下的两个核心发现：（1）RND的不确定性信号——其平方自预测误差——等价于深度集成的预测方差。（2）通过构建特定的RND目标函数，我们证明RND误差分布可以模拟宽神经网络贝叶斯推断的中心后验预测分布。基于此等价性，我们进一步设计了一种后验采样算法，该算法使用改进的\textit{贝叶斯RND}模型从精确的贝叶斯后验预测分布中生成独立同分布样本。总体而言，我们的研究提供了一个统一的理论视角，将RND置于深度集成与贝叶斯推断的原理性框架内，并为高效且理论完备的不确定性量化方法开辟了新途径。