We identify test prediction variance (TPV) -- the first-order sensitivity of model outputs to parameter perturbations around a trained solution -- as a unifying quantity that links several classical observations about generalization in deep networks. TPV is a fully label-free object whose trace form separates the geometry of the trained model from the specific perturbation mechanism, allowing a broad family of parameter perturbations like SGD noise, label noise, finite-precision noise, and other post-training perturbations to be analyzed under a single framework. Theoretically, we show that TPV estimated on the training set converges to its test-set value in the overparameterized limit, providing the first result that prediction variance under local parameter perturbations can be inferred from training inputs alone. Empirically, TPV exhibits a striking stability across datasets and architectures -- including extremely narrow networks -- and correlates well with clean test loss. Finally, we demonstrate that modeling pruning as a TPV perturbation yields a simple label-free importance measure that performs competitively with state-of-the-art pruning methods, illustrating the practical utility of TPV. Code available at github.com/devansharpit/TPV.

翻译：我们提出测试预测方差（TPV）——即模型输出对训练解附近参数扰动的一阶敏感性——作为一个统一量，将深度网络中泛化相关的若干经典观测联系起来。TPV是完全无需标签的度量，其迹形式将训练模型的几何特性与具体扰动机制分离，使得随机梯度下降噪声、标签噪声、有限精度噪声及其他训练后扰动等广泛参数扰动族可在统一框架下分析。理论上，我们证明在过参数化极限下，基于训练集估计的TPV会收敛至其测试集对应值，这首次表明局部参数扰动下的预测方差可仅通过训练输入推断。实证研究表明，TPV在不同数据集和架构（包括极窄网络）中表现出显著稳定性，且与干净测试损失高度相关。最后，我们通过将剪枝建模为TPV扰动，提出一种简单无需标签的重要性度量方法，其性能与最先进剪枝方法相当，体现了TPV的实际应用价值。代码发布于github.com/devansharpit/TPV。