Multi-task learning (MTL) compresses the information from multiple tasks into a unified backbone to improve computational efficiency and generalization. Recent work directly merges multiple independently trained models to perform MTL instead of collecting their raw data for joint training, greatly expanding the application scenarios of MTL. However, by visualizing the representation distribution of existing model merging schemes, we find that the merged model often suffers from the dilemma of representation bias. That is, there is a significant discrepancy in the representation distribution between the merged and individual models, resulting in poor performance of merged MTL. In this paper, we propose a representation surgery solution called "Surgery" to reduce representation bias in the merged model. Specifically, Surgery is a lightweight task-specific module that takes the representation of the merged model as input and attempts to output the biases contained in the representation from the merged model. We then designed an unsupervised optimization objective that updates the Surgery module by minimizing the distance between the merged model's representation and the individual model's representation. Extensive experiments demonstrate significant MTL performance improvements when our Surgery module is applied to state-of-the-art (SOTA) model merging schemes.

翻译：多任务学习（MTL）将多个任务的信息压缩到统一骨干网络中，以提升计算效率和泛化能力。近期研究通过直接融合多个独立训练的模型实现多任务学习，无需收集原始数据进行联合训练，从而极大拓展了MTL的应用场景。然而，通过可视化现有模型融合方案的表示分布，我们发现融合后的模型常面临表示偏差困境：即融合模型与独立模型在表示分布上存在显著差异，导致融合MTL性能欠佳。本文提出一种名为“Surgery”的表示手术解决方案，用于降低融合模型的表示偏差。具体而言，Surgery是一种轻量级任务特定模块，以融合模型的表示为输入，尝试输出融合模型中包含的偏差信息。我们随后设计了一个无监督优化目标，通过最小化融合模型表示与独立模型表示之间的距离来更新Surgery模块。大量实验表明，将我们的Surgery模块应用于最先进的（SOTA）模型融合方案时，可显著提升多任务学习性能。