The last decade has witnessed the success of deep learning and the surge of publicly released trained models, which necessitates the quantification of the model functional distance for various purposes. However, quantifying the model functional distance is always challenging due to the opacity in inner workings and the heterogeneity in architectures or tasks. Inspired by the concept of "field" in physics, in this work we introduce Model Gradient Field (abbr. ModelGiF) to extract homogeneous representations from the heterogeneous pre-trained models. Our main assumption underlying ModelGiF is that each pre-trained deep model uniquely determines a ModelGiF over the input space. The distance between models can thus be measured by the similarity between their ModelGiFs. We validate the effectiveness of the proposed ModelGiF with a suite of testbeds, including task relatedness estimation, intellectual property protection, and model unlearning verification. Experimental results demonstrate the versatility of the proposed ModelGiF on these tasks, with significantly superiority performance to state-of-the-art competitors. Codes are available at https://github.com/zju-vipa/modelgif.

翻译：过去十年见证了深度学习的成功以及公开训练模型的大量涌现，这使得对模型功能距离的量化需求日益增长，以服务于多种目的。然而，由于模型内部运作的不透明性以及架构或任务的异构性，量化模型功能距离始终具有挑战性。受物理学中“场”概念的启发，本文引入了模型梯度场（简称ModelGiF），用于从异构预训练模型中提取同质化表示。ModelGiF的核心假设是：每个预训练的深度模型唯一地决定了一个定义在输入空间上的ModelGiF。因此，模型之间的距离可以通过其ModelGiF之间的相似性来度量。我们通过一系列测试平台验证了所提出的ModelGiF的有效性，包括任务关联性估计、知识产权保护和模型遗忘验证。实验结果表明，所提出的ModelGiF在这些任务上具有广泛的适用性，其性能显著优于当前最先进的竞争对手。代码已公开于https://github.com/zju-vipa/modelgif。