Multi-output deep neural networks(MONs) contain multiple task branches, and these tasks usually share partial network filters that lead to the entanglement of different task inference routes. Due to the inconsistent optimization objectives, the task gradients used for training MONs will interfere with each other on the shared routes, which will decrease the overall model performance. To address this issue, we propose a novel gradient de-conflict algorithm named DR-MGF(Dynamic Routes and Meta-weighted Gradient Fusion) in this work. Different from existing de-conflict methods, DR-MGF achieves gradient de-conflict in MONs by learning task-preferred inference routes. The proposed method is motivated by our experimental findings: the shared filters are not equally important to different tasks. By designing the learnable task-specific importance variables, DR-MGF evaluates the importance of filters for different tasks. Through making the dominances of tasks over filters be proportional to the task-specific importance of filters, DR-MGF can effectively reduce the inter-task interference. The task-specific importance variables ultimately determine task-preferred inference routes at the end of training iterations. Extensive experimental results on CIFAR, ImageNet, and NYUv2 illustrate that DR-MGF outperforms the existing de-conflict methods both in prediction accuracy and convergence speed of MONs. Furthermore, DR-MGF can be extended to general MONs without modifying the overall network structures.

翻译：多输出深度神经网络包含多个任务分支，这些任务通常共享部分网络滤波器，导致不同任务推理路径的相互纠缠。由于优化目标的不一致性，用于训练多输出网络的各任务梯度会在共享路径上产生相互干扰，从而降低整体模型性能。针对这一问题，本文提出一种名为DR-MGF（动态路径与元加权梯度融合）的新型梯度冲突消除算法。与现有去冲突方法不同，DR-MGF通过学习任务偏好的推理路径来实现多输出网络的梯度冲突消除。该方法的提出源于我们的实验发现：共享滤波器对不同任务的重要性并不相同。通过设计可学习的任务特异性重要性变量，DR-MGF能够评估不同任务的滤波器重要性。通过使任务对滤波器的主导权与任务特异性滤波器重要性成正比，DR-MGF可有效降低任务间干扰。在训练迭代结束时，任务特异性重要性变量最终确定任务偏好的推理路径。在CIFAR、ImageNet和NYUv2数据集上的大量实验结果表明，DR-MGF在多输出网络的预测精度和收敛速度方面均优于现有去冲突方法。此外，DR-MGF可在不修改整体网络结构的情况下扩展至通用多输出网络。