Good models require good training data. For overparameterized deep models, the causal relationship between training data and model predictions is increasingly opaque and poorly understood. Influence analysis partially demystifies training's underlying interactions by quantifying the amount each training instance alters the final model. Measuring the training data's influence exactly can be provably hard in the worst case; this has led to the development and use of influence estimators, which only approximate the true influence. This paper provides the first comprehensive survey of training data influence analysis and estimation. We begin by formalizing the various, and in places orthogonal, definitions of training data influence. We then organize state-of-the-art influence analysis methods into a taxonomy; we describe each of these methods in detail and compare their underlying assumptions, asymptotic complexities, and overall strengths and weaknesses. Finally, we propose future research directions to make influence analysis more useful in practice as well as more theoretically and empirically sound. A curated, up-to-date list of resources related to influence analysis is available at https://github.com/ZaydH/influence_analysis_papers.

翻译：优秀的模型需要优质的训练数据。对于过参数化的深度模型而言，训练数据与模型预测之间的因果关系正变得愈发模糊且难以理解。影响力分析通过量化每个训练样本对最终模型的影响程度，部分揭示了训练过程中潜在的交互机制。在最坏情况下，精确测量训练数据的影响力被证明是困难的；这促使了仅近似真实影响力的估计器的开发与应用。本文首次对训练数据影响力分析与估计进行了全面综述。我们首先形式化了训练数据影响力的多种（部分情况下正交的）定义。随后，我们将当前最先进的影响力分析方法组织成分类体系；详细描述每种方法，并比较其底层假设、渐近复杂度以及总体优缺点。最后，我们提出了未来研究方向，旨在使影响力分析在实践中更具实用性，同时在理论和经验层面更加可靠。与影响力分析相关的精选最新资源列表可访问 https://github.com/ZaydH/influence_analysis_papers。