Model merging combines knowledge from separately fine-tuned models, yet the factors driving its success remain poorly understood. While recent work treats mergeability as an intrinsic property of the models, we show with an architecture-agnostic framework that it fundamentally depends on both the merging method and the partner tasks. Using L1-regularized linear optimization over a set of interpretable pairwise metrics (e.g., gradient $L_2$ distance), we uncover properties correlating with post-merge normalized accuracy across five merging methods. We find architecture- and method-specific variation in success drivers (64.0% average top-5 metric overlap; 79.3% sign agreement), with certain methods, notably TIES, exhibiting distinct ``fingerprints'' that diverge from the broader consensus. Crucially, however, \textit{gradient alignment} metrics consistently emerge as the most fundamental signals of compatibility. These findings provide a diagnostic foundation for understanding mergeability and motivate future merge-aware fine-tuning strategies.
翻译:模型融合结合了分别微调模型的知识,但决定其成功的关键因素尚不明确。尽管近期研究将融合性视为模型的内在属性,我们通过一个与架构无关的框架证明,融合性从根本上取决于融合方法和伙伴任务。基于一组可解释的成对度量指标(例如梯度$L_2$距离)的L1正则化线性优化,我们揭示了五种融合方法中与融合后归一化准确率相关的属性。我们发现成功驱动因素存在架构特异性和方法特异性(平均前5个度量重叠率64.0%,符号一致性79.3%),其中某些方法(特别是TIES)表现出偏离主流共识的独特“指纹”。然而关键的是,\textit{梯度对齐}度量始终是兼容性最基础的信号。这些发现为理解融合性提供了诊断基础,并启发了未来融合感知的微调策略。