A recent trend in explainable AI research has focused on surrogate modeling, where neural networks are approximated as simpler ML algorithms such as kernel machines. A second trend has been to utilize kernel functions in various explain-by-example or data attribution tasks to investigate a diverse set of neural network behavior. In this work, we combine these two trends to analyze approximate empirical neural tangent kernels (eNTK) for data attribution. Approximation is critical for eNTK analysis due to the high computational cost to compute the eNTK. We define new approximate eNTK and perform novel analysis on how well the resulting kernel machine surrogate models correlate with the underlying neural network. We introduce two new random projection variants of approximate eNTK which allow users to tune the time and memory complexity of their calculation. We conclude that kernel machines using approximate neural tangent kernel as the kernel function are effective surrogate models, with the introduced trace NTK the most consistent performer.

翻译：可解释人工智能研究的一个近期趋势聚焦于替代建模，即通过更简单的机器学习算法（如核方法）来近似神经网络。第二个趋势是利用核函数进行各种“通过示例解释”或数据归因任务，以探究神经网络行为的多样性。本研究结合这两个趋势，分析用于数据归因的近似经验神经切线核（eNTK）。由于计算eNTK的高昂计算成本，近似分析至关重要。我们定义了新的近似eNTK，并创新性地分析了由此产生的核方法替代模型与原始神经网络的关联程度。我们引入了两种新的近似eNTK随机投影变体，使用户能够调节其计算的时间与内存复杂度。结论表明，采用近似神经切线核作为核函数的核方法是有效的替代模型，其中引入的迹NTK表现最为稳定。