We identify test prediction variance (TPV)-- the first-order sensitivity of model outputs to parameter perturbations around a trained solution-- as a unifying quantity that links several classical observations about generalization in deep networks. TPV is a fully label-free object whose trace form separates the geometry of the trained model from the specific perturbation mechanism, allowing a broad family of parameter perturbations like SGD noise, label noise, finite-precision noise, and other post-training perturbations to be analyzed under a single framework. Theoretically, we show that TPV estimated on the training set converges to its test-set value in the overparameterized limit, providing the first result that prediction variance under local parameter perturbations can be inferred from training inputs alone, and this stability is decoupled from generalization performance. Empirically, TPV exhibits a striking stability across datasets and architectures even for extremely narrow networks. Further, TPV correlates well with test loss, serving as a training-set based predictive metric for generalization. Code available at github.com/devansharpit/TPV.

翻译：我们提出测试预测方差（TPV）——即模型输出对训练解附近参数扰动的一阶敏感性——作为连接深度网络泛化中若干经典观测的统一量。TPV是完全无标签的度量对象，其迹形式将训练模型的几何特性与具体扰动机制分离，使得随机梯度下降噪声、标签噪声、有限精度噪声及其他训练后扰动等广泛参数扰动族能在统一框架下分析。理论上，我们证明在过参数化极限下，基于训练集估计的TPV收敛于其测试集对应值，首次实现了局部参数扰动下的预测方差可仅通过训练输入推断，且该稳定性与泛化性能解耦。实验表明，TPV在不同数据集和架构间（包括极窄网络）均表现出显著稳定性。此外，TPV与测试损失高度相关，可作为基于训练集的泛化预测指标。代码发布于github.com/devansharpit/TPV。