While multi-task learning (MTL) has gained significant attention in recent years, its underlying mechanisms remain poorly understood. Recent methods did not yield consistent performance improvements over single task learning (STL) baselines, underscoring the importance of gaining more profound insights about challenges specific to MTL. In our study, we challenge common assumptions in MTL in the context of STL: First, the choice of optimizer has only been mildly investigated in MTL. We show the pivotal role of common STL tools such as the Adam optimizer in MTL. We deduce the effectiveness of Adam to its partial loss-scale invariance. Second, the notion of gradient conflicts has often been phrased as a specific problem in MTL. We delve into the role of gradient conflicts in MTL and compare it to STL. For angular gradient alignment we find no evidence that this is a unique problem in MTL. We emphasize differences in gradient magnitude as the main distinguishing factor. Lastly, we compare the transferability of features learned through MTL and STL on common image corruptions, and find no conclusive evidence that MTL leads to superior transferability. Overall, we find surprising similarities between STL and MTL suggesting to consider methods from both fields in a broader context.

翻译：近年来，多任务学习虽备受关注，但其内在机制仍未被充分理解。现有方法相较于单任务学习基线未取得一致性能提升，凸显了对多任务学习特有挑战进行深入剖析的重要性。本研究在单任务学习框架下对多任务学习中的常见假设提出了质疑：首先，优化器的选择在多任务学习中鲜少被系统探究，我们揭示了Adam等单任务学习常用工具在多任务学习中的关键作用，并将其有效性归因于其部分损失尺度不变性。其次，梯度冲突通常被视为多任务学习的特有问题，我们深入探讨了梯度冲突在多任务学习中的角色，并与单任务学习进行了比较。对于角度梯度对齐，未发现该现象是多任务学习的独有问题，而梯度幅值差异才是主要区分因素。最后，我们对比了多任务学习与单任务学习在常见图像扰动下特征的可迁移性，未获得多任务学习具有优越可迁移性的确凿证据。总体而言，我们发现单任务学习与多任务学习存在惊人相似性，建议在更广泛背景下综合考量两个领域的方法。