Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. Oftentimes fine-tuned models are readily available but their training data is not, due to data privacy or intellectual property concerns. This creates a barrier to fusing knowledge across individual models to yield a better single model. In this paper, we study the problem of merging individual models built on different training data sets to obtain a single model that performs well both across all data set domains and can generalize on out-of-domain data. We propose a dataless knowledge fusion method that merges models in their parameter space, guided by weights that minimize prediction differences between the merged model and the individual models. Over a battery of evaluation settings, we show that the proposed method significantly outperforms baselines such as Fisher-weighted averaging or model ensembling. Further, we find that our method is a promising alternative to multi-task learning that can preserve or sometimes improve over the individual models without access to the training data. Finally, model merging is more efficient than training a multi-task model, thus making it applicable to a wider set of scenarios.

翻译：微调预训练语言模型已成为构建下游NLP模型的主流范式。尽管微调后的模型通常易于获取，但由于数据隐私或知识产权等问题，其训练数据往往不可得。这为融合单个模型的知识以获得更优的单一模型设置了障碍。本文研究融合基于不同训练数据集构建的个体模型的问题，旨在获得一个既能覆盖所有数据集领域性能、又能泛化至域外数据的单一模型。我们提出了一种无数据知识融合方法，该方法在参数空间中合并模型，通过最小化合并模型与个体模型之间预测差异的权重进行引导。在一系列评估设置中，我们证明了所提方法显著优于Fisher加权平均或模型集成等基线方法。此外，我们发现该方法可作为多任务学习的有前景替代方案，能够在无需访问训练数据的情况下保持甚至有时提升个体模型性能。最后，模型合并比训练多任务模型更高效，因而适用于更广泛的场景。