Understanding model's sensitivity to its training data is crucial but can also be challenging and costly, especially during training. To simplify such issues, we present the Memory-Perturbation Equation (MPE) which relates model's sensitivity to perturbation in its training data. Derived using Bayesian principles, the MPE unifies existing sensitivity measures, generalizes them to a wide-variety of models and algorithms, and unravels useful properties regarding sensitivities. Our empirical results show that sensitivity estimates obtained during training can be used to faithfully predict generalization on unseen test data. The proposed equation is expected to be useful for future research on robust and adaptive learning.

翻译：理解模型对训练数据的敏感性至关重要，但在训练过程中这一过程往往充满挑战且代价高昂。为简化此类问题，我们提出记忆-扰动方程（Memory-Perturbation Equation, MPE），该方程将模型敏感性与其训练数据扰动相关联。MPE基于贝叶斯原理推导，统一了现有敏感性度量方法，并将其推广至各类模型与算法，同时揭示了敏感性的实用性质。实验结果表明，训练过程中获得的敏感性估计值可有效预测模型在未见测试数据上的泛化能力。该方程预计将对未来鲁棒性与自适应学习研究具有重要价值。