Similarity, or clone, detection has important applications in copyright violation, software theft, code search, and the detection of malicious components. There is now a good number of open source and proprietary clone detectors for programs written in traditional programming languages. However, the increasing adoption of deep learning models in software poses a challenge to these tools: these models implement functions that are inscrutable black boxes. As more software includes these DNN functions, new techniques are needed in order to assess the similarity between deep learning components of software. Previous work has unveiled techniques for comparing the representations learned at various layers of deep neural network models by feeding canonical inputs to the models. Our goal is to be able to compare DNN functions when canonical inputs are not available -- because they may not be in many application scenarios. The challenge, then, is to generate appropriate inputs and to identify a metric that, for those inputs, is capable of representing the degree of functional similarity between two comparable DNN functions. Our approach uses random input with values between -1 and 1, in a shape that is compatible with what the DNN models expect. We then compare the outputs by performing correlation analysis. Our study shows how it is possible to perform similarity analysis even in the absence of meaningful canonical inputs. The response to random inputs of two comparable DNN functions exposes those functions' similarity, or lack thereof. Of all the metrics tried, we find that Spearman's rank correlation coefficient is the most powerful and versatile, although in special cases other methods and metrics are more expressive. We present a systematic empirical study comparing the effectiveness of several similarity metrics using a dataset of 56,355 classifiers collected from GitHub. This is accompanied by a sensitivity analysis that reveals how certain models' training related properties affect the effectiveness of the similarity metrics. To the best of our knowledge, this is the first work that shows how similarity of DNN functions can be detected by using random inputs. Our study of correlation metrics, and the identification of Spearman correlation coefficient as the most powerful among them for this purpose, establishes a complete and practical method for DNN clone detection that can be used in the design of new tools. It may also serve as inspiration for other program analysis tasks whose approaches break in the presence of DNN components.

翻译：相似性（或称克隆）检测在版权侵权、软件盗窃、代码搜索以及恶意组件检测中具有重要应用。目前，针对传统编程语言编写的程序，已有相当数量的开源与专有克隆检测工具。然而，深度学习模型在软件中的日益普及对这些工具提出了挑战：这些模型实现的功能如同难以解读的黑箱。随着越来越多的软件包含这些深度神经网络（DNN）功能，需要开发新技术来评估软件中深度学习组件的相似性。先前的研究通过向模型输入规范数据，揭示了比较深度神经网络模型各层所学表征的技术。我们的目标是在缺乏规范数据（这在许多应用场景中可能不存在）的情况下，比较DNN函数的相似性。因此，挑战在于生成合适的输入，并确定一种能够针对这些输入，表征两个可比较DNN函数之间功能相似程度的度量方法。我们的方法使用值域在-1到1之间的随机输入，其形状与DNN模型期望的输入兼容。随后，我们通过相关性分析对输出进行比较。我们的研究表明，即使在缺乏有意义规范数据的情况下，也能进行相似性分析。两个可比较DNN函数对随机输入的响应，能够揭示这些函数的相似性（或缺乏相似性）。在尝试的所有度量方法中，我们发现斯皮尔曼等级相关系数是最强大且最通用的，尽管在特殊情况下其他方法和度量更具表现力。我们基于从GitHub收集的56,355个分类器数据集，进行了一项系统性实证研究，比较了多种相似性度量的有效性。同时，通过敏感性分析揭示了某些模型的训练相关属性如何影响相似性度量的有效性。据我们所知，这是首个展示如何通过随机输入检测DNN函数相似性的工作。我们对相关性度量的研究，以及将斯皮尔曼相关系数确定为该目的下最强大的度量，建立了一套完整且实用的DNN克隆检测方法，可用于新工具的设计。这项工作也可能启发其他在面对DNN组件时失效的程序分析任务。