Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. Oftentimes fine-tuned models are readily available but their training data is not, due to data privacy or intellectual property concerns. This creates a barrier to fusing knowledge across individual models to yield a better single model. In this paper, we study the problem of merging individual models built on different training data sets to obtain a single model that performs well both across all data set domains and can generalize on out-of-domain data. We propose a dataless knowledge fusion method that merges models in their parameter space, guided by weights that minimize prediction differences between the merged model and the individual models. Over a battery of evaluation settings, we show that the proposed method significantly outperforms baselines such as Fisher-weighted averaging or model ensembling. Further, we find that our method is a promising alternative to multi-task learning that can preserve or sometimes improve over the individual models without access to the training data. Finally, model merging is more efficient than training a multi-task model, thus making it applicable to a wider set of scenarios.

翻译：微调预训练语言模型已成为构建下游自然语言处理模型的主流范式。尽管微调后的模型通常易于获取，但其训练数据往往因数据隐私或知识产权问题而无法公开。这一现状阻碍了通过融合多个独立模型的知識来构建性能更优的单一模型。本文研究了如何合并基于不同训练数据集构建的独立模型，从而获得一个既能全面覆盖各数据集领域、又能在域外数据上具备泛化能力的单一模型。我们提出一种无需数据的知识融合方法，该方法在参数空间中对模型进行合并，通过权重引导使合并模型与各独立模型之间的预测差异最小化。在多项评估设置中，我们证明该方法显著优于Fisher加权平均或模型集成等基线方法。此外，我们的方法可作为多任务学习的有效替代方案，无需访问训练数据即可保持甚至提升独立模型的性能。最后，模型合并相较于训练多任务模型具有更高的计算效率，使其适用于更广泛的场景。