Multitask Learning is a Machine Learning paradigm that aims to train a range of (usually related) tasks with the help of a shared model. While the goal is often to improve the joint performance of all training tasks, another approach is to focus on the performance of a specific target task, while treating the remaining ones as auxiliary data from which to possibly leverage positive transfer towards the target during training. In such settings, it becomes important to estimate the positive or negative influence auxiliary tasks will have on the target. While many ways have been proposed to estimate task weights before or during training they typically rely on heuristics or extensive search of the weighting space. We propose a novel method called $\alpha$-Variable Importance Learning ($\alpha$VIL) that is able to adjust task weights dynamically during model training, by making direct use of task-specific updates of the underlying model's parameters between training epochs. Experiments indicate that $\alpha$VIL is able to outperform other Multitask Learning approaches in a variety of settings. To our knowledge, this is the first attempt at making direct use of model updates for task weight estimation.

翻译：多任务学习是一种机器学习范式，旨在通过共享模型训练一系列（通常是相关的）任务。虽然目标通常是提升所有训练任务的联合性能，但另一种方法是专注于特定目标任务的性能，同时将其他任务视为辅助数据，在训练过程中可能利用这些数据向目标任务进行正迁移。在这种设置下，评估辅助任务对目标任务的正向或负向影响变得至关重要。虽然已有多种方法被提出用于在训练前或训练中估算任务权重，但它们通常依赖于启发式方法或对权重空间的广泛搜索。我们提出了一种名为α-变量重要性学习（αVIL）的新方法，该方法通过直接利用训练轮次之间底层模型参数的任务特定更新，能够在模型训练过程中动态调整任务权重。实验表明，αVIL在各种设置中均能优于其他多任务学习方法。据我们所知，这是首次尝试直接利用模型更新来估算任务权重。