Multi-task learning (MTL) seeks to learn a single model to accomplish multiple tasks by leveraging shared information among the tasks. Existing MTL models, however, have been known to suffer from negative interference among tasks. Efforts to mitigate task interference have focused on either loss/gradient balancing or implicit parameter partitioning with partial overlaps among the tasks. In this paper, we propose ETR-NLP to mitigate task interference through a synergistic combination of non-learnable primitives (NLPs) and explicit task routing (ETR). Our key idea is to employ non-learnable primitives to extract a diverse set of task-agnostic features and recombine them into a shared branch common to all tasks and explicit task-specific branches reserved for each task. The non-learnable primitives and the explicit decoupling of learnable parameters into shared and task-specific ones afford the flexibility needed for minimizing task interference. We evaluate the efficacy of ETR-NLP networks for both image-level classification and pixel-level dense prediction MTL problems. Experimental results indicate that ETR-NLP significantly outperforms state-of-the-art baselines with fewer learnable parameters and similar FLOPs across all datasets. Code is available at this \href{https://github.com/zhichao-lu/etr-nlp-mtl}.

翻译：多任务学习旨在通过利用任务间的共享信息，学习一个单一模型以完成多个任务。然而，现有研究表明，多任务模型常遭受任务间的负面干扰。缓解任务干扰的努力主要集中在损失/梯度平衡或任务间部分重叠的隐式参数划分上。本文提出ETR-NLP方法，通过非可学习基元与显式任务路由的协同组合来缓解任务干扰。我们的核心思路是：采用非可学习基元提取多样化的任务无关特征，将其重组为所有任务共用的共享分支，并为每个任务保留显式的任务专用分支。非可学习基元以及将可学习参数显式解耦为共享参数和任务专用参数，为最小化任务干扰提供了所需的灵活性。我们针对图像级分类和像素级密集预测两种多任务学习问题，评估了ETR-NLP网络的效能。实验结果表明，ETR-NLP在所有数据集上以更少的可学习参数和相当的FLOPs显著优于当前最优基线方法。代码已开源发布于 \href{https://github.com/zhichao-lu/etr-nlp-mtl}。