Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. Oftentimes fine-tuned models are readily available but their training data is not, due to data privacy or intellectual property concerns. This creates a barrier to fusing knowledge across individual models to yield a better single model. In this paper, we study the problem of merging individual models built on different training data sets to obtain a single model that performs well both across all data set domains and can generalize on out-of-domain data. We propose a dataless knowledge fusion method that merges models in their parameter space, guided by weights that minimize prediction differences between the merged model and the individual models. Over a battery of evaluation settings, we show that the proposed method significantly outperforms baselines such as Fisher-weighted averaging or model ensembling. Further, we find that our method is a promising alternative to multi-task learning that can preserve or sometimes improve over the individual models without access to the training data. Finally, model merging is more efficient than training a multi-task model, thus making it applicable to a wider set of scenarios.

翻译：微调预训练语言模型已成为构建下游NLP模型的主流范式。尽管微调后的模型通常易于获取，但由于数据隐私或知识产权问题，其训练数据往往不可得。这阻碍了通过融合多个独立模型的知识来获得更优单一模型的过程。本文研究了如何合并基于不同训练数据集构建的独立模型，以得到一个既能良好处理所有数据集领域任务、又能泛化至域外数据的单一模型。我们提出了一种无需数据知识融合方法，该方法在参数空间中通过权重引导合并模型，这些权重优化了合并模型与各独立模型预测差异的最小化。在多种评估设置下，我们证明了所提方法显著优于Fisher加权平均或模型集成等基线方法。此外，我们发现该方法可作为多任务学习的有效替代方案，在无需访问训练数据的情况下保持甚至提升独立模型的性能。最后，模型合并相比训练多任务模型更具效率，因此可适用于更广泛的场景。