Influence functions offer a principled way to trace model predictions back to training data, but their use in deep learning is hampered by the need to invert a large, ill-conditioned Hessian matrix. Approximations such as Generalised Gauss-Newton (GGN) and Kronecker-Factored Approximate Curvature (K-FAC) have been proposed to make influence computation tractable, yet it remains unclear how the departure from exactness impacts data attribution performance. Critically, given the restricted regime in which influence functions are derived, it is not necessarily clear better Hessian approximations should even lead to better data attribution performance. In this paper, we investigate the effect of Hessian approximation quality on influence-function attributions in a controlled classification setting. Our experiments show that better Hessian approximations consistently yield better influence score quality, offering justification for recent research efforts towards that end. We further decompose the approximation steps for recent Hessian approximation methods and evaluate each step's influence on attribution accuracy. Notably, the mismatch between K-FAC eigenvalues and GGN/EK-FAC eigenvalues accounts for the majority of the error and influence loss. These findings highlight which approximations are most critical, guiding future efforts to balance computational tractability and attribution accuracy.

翻译：影响函数为追溯模型预测至训练数据提供了理论方法，但其在深度学习中的应用受限于需要求逆大型病态Hessian矩阵的难题。广义高斯-牛顿（GGN）和克罗内克分解近似曲率（K-FAC）等近似方法已被提出以使影响计算可行，然而这些近似与精确解的偏离如何影响数据归因性能仍不明确。关键在于，考虑到影响函数的推导建立在受限的理论框架下，更好的Hessian近似是否必然带来更优的数据归因性能尚存疑问。本文在受控分类场景下系统研究了Hessian近似质量对影响函数归因效果的影响。实验表明，更精确的Hessian近似始终能产生更高质量的影响分数，这为近期相关研究方向提供了实证依据。我们进一步解构了当前主流Hessian近似方法的计算步骤，并评估了每个步骤对归因准确性的影响。值得注意的是，K-FAC特征值与GGN/EK-FAC特征值之间的差异构成了误差和影响损失的主要来源。这些发现明确了哪些近似步骤最为关键，为未来平衡计算可行性与归因准确性的研究提供了方向指引。