We identify test prediction variance (TPV) -- the first-order sensitivity of model outputs to parameter perturbations around a trained solution -- as a unifying quantity that links several classical observations about generalization in deep networks. TPV is a fully label-free object whose trace form separates the geometry of the trained model from the specific perturbation mechanism, allowing a broad family of parameter perturbations like SGD noise, label noise, finite-precision noise, and other post-training perturbations to be analyzed under a single framework. Theoretically, we show that TPV estimated on the training set converges to its test-set value in the overparameterized limit, providing the first result that prediction variance under local parameter perturbations can be inferred from training inputs alone. Empirically, TPV exhibits a striking stability across datasets and architectures -- including extremely narrow networks -- and correlates well with clean test loss. Finally, we demonstrate that modeling pruning as a TPV perturbation yields a simple label-free importance measure that performs competitively with state-of-the-art pruning methods, illustrating the practical utility of TPV. Code available at github.com/devansharpit/TPV.

翻译：我们提出测试预测方差（TPV）——即模型输出对训练解附近参数扰动的一阶敏感性——作为一个统一量，它连接了深度网络中关于泛化的若干经典观察。TPV是一种完全无需标签的对象，其迹形式将训练模型的几何结构与特定扰动机制分离，使得包括SGD噪声、标签噪声、有限精度噪声以及其他训练后扰动在内的广泛参数扰动族可在单一框架下分析。理论上，我们证明在过参数化极限下，基于训练集估计的TPV收敛于其测试集值，这首次表明局部参数扰动下的预测方差可仅从训练输入中推断。实证上，TPV在数据集和架构（包括极窄网络）间表现出显著的稳定性，并与干净测试损失良好相关。最后，我们通过将剪枝建模为TPV扰动，提出一种简单的无需标签的重要性度量，其性能与最先进的剪枝方法相当，从而展示了TPV的实际应用价值。代码发布于github.com/devansharpit/TPV。