Fine-tuning pre-trained language models has become the prevalent paradigm for building downstream NLP models. Oftentimes fine-tuned models are readily available but their training data is not, due to data privacy or intellectual property concerns. This creates a barrier to fusing knowledge across individual models to yield a better single model. In this paper, we study the problem of merging individual models built on different training data sets to obtain a single model that performs well both across all data set domains and can generalize on out-of-domain data. We propose a dataless knowledge fusion method that merges models in their parameter space, guided by weights that minimize prediction differences between the merged model and the individual models. Over a battery of evaluation settings, we show that the proposed method significantly outperforms baselines such as Fisher-weighted averaging or model ensembling. Further, we find that our method is a promising alternative to multi-task learning that can preserve or sometimes improve over the individual models without access to the training data. Finally, model merging is more efficient than training a multi-task model, thus making it applicable to a wider set of scenarios.

翻译：在预训练语言模型上进行微调已成为构建下游NLP模型的普遍范式。尽管微调后的模型通常易于获取，但受数据隐私或知识产权保护的限制，其训练数据往往不可得。这给通过融合多个独立模型的知识以获得更优单模型带来了障碍。本文研究如何合并基于不同训练数据集构建的独立模型，从而获得一个既能覆盖所有数据集领域、又能对域外数据进行泛化的统一模型。我们提出一种无数据知识融合方法，该方法在参数空间内合并模型，通过最小化合并模型与独立模型间预测差异的权重引导实现融合。在多项评估实验中，我们证明该方法显著优于费舍尔加权平均或模型集成等基线方法。进一步研究表明，该方法是多任务学习的有效替代方案，可在无需访问训练数据的前提下保留甚至提升独立模型的性能。最后，模型合并比训练多任务模型更具效率，因而可适用于更广泛的场景。