Measuring similarity of neural networks has become an issue of great importance and research interest to understand and utilize differences of neural networks. While there are several perspectives on how neural networks can be similar, we specifically focus on two complementing perspectives, i.e., (i) representational similarity, which considers how activations of intermediate neural layers differ, and (ii) functional similarity, which considers how models differ in their outputs. In this survey, we provide a comprehensive overview of these two families of similarity measures for neural network models. In addition to providing detailed descriptions of existing measures, we summarize and discuss results on the properties and relationships of these measures, and point to open research problems. Further, we provide practical recommendations that can guide researchers as well as practitioners in applying the measures. We hope our work lays a foundation for our community to engage in more systematic research on the properties, nature and applicability of similarity measures for neural network models.

翻译：测量神经网络的相似性已成为理解和利用神经网络差异的重要研究课题。虽然关于神经网络相似性的考察存在多种视角，我们聚焦于两个互补的角度：（i）表征相似性，即考察中间神经层激活模式的差异；（ii）功能相似性，即考察模型输出结果的差异。本综述系统梳理了这两类神经网络模型相似性测量方法。除详细阐述现有测量方法外，我们总结并讨论了这些方法在属性特征与相互关系方面的研究成果，指出了有待解决的研究问题。同时，我们为研究人员和从业者提供了应用这些测量方法的实用建议。希望本工作能为学术共同体开展关于神经网络模型相似性测量方法的属性、本质与适用性的系统性研究奠定基础。