A recent trend in explainable AI research has focused on surrogate modeling, where neural networks are approximated as simpler ML algorithms such as kernel machines. A second trend has been to utilize kernel functions in various explain-by-example or data attribution tasks to investigate a diverse set of neural network behavior. In this work, we combine these two trends to analyze approximate empirical neural tangent kernels (eNTK) for data attribution. Approximation is critical for eNTK analysis due to the high computational cost to compute the eNTK. We define new approximate eNTK and perform novel analysis on how well the resulting kernel machine surrogate models correlate with the underlying neural network. We introduce two new random projection variants of approximate eNTK which allow users to tune the time and memory complexity of their calculation. We conclude that kernel machines using approximate neural tangent kernel as the kernel function are effective surrogate models, with the introduced trace NTK the most consistent performer.

翻译：可解释人工智能研究的最新趋势聚焦于代理建模，即通过核机器等更简单的机器学习算法来近似神经网络。另一趋势是利用各类核函数进行基于示例的解释或数据归因任务，以探究神经网络的多样化行为。本研究将这两种趋势相结合，分析近似经验神经正切核（eNTK）在数据归因中的应用。由于计算eNTK的计算成本高昂，近似方法对eNTK分析至关重要。我们定义了新的近似eNTK，并开展了关于所得核机器代理模型与底层神经网络相关性的创新分析。我们引入了两种新的近似eNTK随机投影变体，使用户能够调节其计算的时间与内存复杂度。最终得出结论：以近似神经正切核作为核函数的核机器是有效的代理模型，其中我们提出的迹NTK性能最为稳定。