With the rise of neural networks in various domains, multi-task learning (MTL) gained significant relevance. A key challenge in MTL is balancing individual task losses during neural network training to improve performance and efficiency through knowledge sharing across tasks. To address these challenges, we propose a novel task-weighting method by building on the most prevalent approach of Uncertainty Weighting and computing analytically optimal uncertainty-based weights, normalized by a softmax function with tunable temperature. Our approach yields comparable results to the combinatorially prohibitive, brute-force approach of Scalarization while offering a more cost-effective yet high-performing alternative. We conduct an extensive benchmark on various datasets and architectures. Our method consistently outperforms six other common weighting methods. Furthermore, we report noteworthy experimental findings for the practical application of MTL. For example, larger networks diminish the influence of weighting methods, and tuning the weight decay has a low impact compared to the learning rate.

翻译：随着神经网络在各个领域的兴起，多任务学习（MTL）获得了显著关注。MTL中的一个关键挑战是在神经网络训练过程中平衡各任务的损失，以通过任务间的知识共享来提高性能和效率。为应对这些挑战，我们提出了一种新颖的任务加权方法。该方法基于最主流的不确定性加权方法，通过解析计算基于不确定性的最优权重，并使用具有可调温度参数的softmax函数进行归一化。我们的方法取得了与计算组合上不可行的标量化暴力穷举法相当的结果，同时提供了一种更具成本效益且高性能的替代方案。我们在多种数据集和架构上进行了广泛的基准测试。我们的方法始终优于其他六种常见的加权方法。此外，我们报告了关于MTL实际应用的一些值得注意的实验发现。例如，更大的网络会削弱加权方法的影响，并且与学习率相比，调整权重衰减的影响较小。