Software reuse has long been recognized as a critical and widely studied topic in software engineering, offering substantial benefits in reducing development costs, improving software quality, and enhancing operational efficiency. This paradigm extends into deep learning through model reuse. Recently, model merging has emerged in the domain of large language models (LLMs) as a training-free approach that takes multiple task-specific models with the same architecture as source models and merges them without retraining, enhancing model reuse within LLMs. However, no prior work has systematically investigated whether such an approach can be effectively applied to other deep learning models with different architectures across domains. To bridge this gap, we present the first systematic study that evaluates five model merging techniques on three distinct model architectures across three domains: LLMs, image classification, and autonomous driving. Our findings reveal that directly applying existing model merging techniques leads to highly inconsistent results and falls notably short of their success within LLMs. Moreover, a single model merging technique often fails to handle the heterogeneous structural properties within a model, limiting its applicability to different model architectures across domains. Furthermore, the effectiveness of model merging techniques is highly sensitive to hyperparameter configurations, thereby constraining their potential for broader adoption. Inspired by these insights, we propose AutoMerge, a novel search-based model merging framework that first segments complex models into multiple heterogeneous blocks and then systematically explores the merging space to identify the merging technique and its hyperparameter configuration.

翻译：软件复用长期以来被公认为软件工程中一个关键且被广泛研究的课题，其在降低开发成本、提升软件质量以及增强运行效率方面具有显著优势。这一范式通过模型复用扩展到了深度学习领域。最近，在大语言模型领域兴起了模型融合技术，作为一种免训练的方法，它以多个具有相同架构的特定任务模型作为源模型，无需重新训练即可将其融合，从而提升大语言模型内部的模型复用效率。然而，尚无先前研究工作系统性地探讨此类方法是否能有效应用于跨领域、具有不同架构的其他深度学习模型。为填补这一空白，我们首次开展了系统性研究，在三个不同领域（大语言模型、图像分类和自动驾驶）的三种不同模型架构上评估了五种模型融合技术。我们的研究结果表明，直接应用现有的模型融合技术会导致结果高度不一致，且远未达到其在大语言模型中所取得的成功。此外，单一的模型融合技术往往难以处理模型内部异构的结构特性，从而限制了其在不同领域、不同模型架构上的适用性。更重要的是，模型融合技术的有效性对超参数配置高度敏感，这进一步制约了其更广泛应用的潜力。基于这些发现，我们提出了AutoMerge，一种新颖的基于搜索的模型融合框架。该框架首先将复杂模型分割为多个异构模块，然后系统性地探索融合空间，以确定最佳的融合技术及其超参数配置。