While multi-task learning (MTL) has gained significant attention in recent years, its underlying mechanisms remain poorly understood. Recent methods did not yield consistent performance improvements over single task learning (STL) baselines, underscoring the importance of gaining more profound insights about challenges specific to MTL. In our study, we challenge common assumptions in MTL in the context of STL: First, the choice of optimizer has only been mildly investigated in MTL. We show the pivotal role of common STL tools such as the Adam optimizer in MTL. We deduce the effectiveness of Adam to its partial loss-scale invariance. Second, the notion of gradient conflicts has often been phrased as a specific problem in MTL. We delve into the role of gradient conflicts in MTL and compare it to STL. For angular gradient alignment we find no evidence that this is a unique problem in MTL. We emphasize differences in gradient magnitude as the main distinguishing factor. Lastly, we compare the transferability of features learned through MTL and STL on common image corruptions, and find no conclusive evidence that MTL leads to superior transferability. Overall, we find surprising similarities between STL and MTL suggesting to consider methods from both fields in a broader context.

翻译：虽然多任务学习近年来受到广泛关注，但其内在机制仍未被充分理解。近期方法未能取得优于单任务学习基线的稳定性能提升，这凸显了深入理解多任务学习特有挑战的重要性。本研究从单任务学习视角挑战多任务学习的常见假设：第一，优化器的选择在多任务学习中尚未得到充分探究。我们证明了Adam优化器等常见单任务学习工具在多任务学习中的关键作用，并将其有效性归因于其部分损失尺度不变性。第二，梯度冲突常被视作多任务学习中的特定问题。我们深入探究梯度冲突在多任务学习中的作用，并与单任务学习进行对比。针对角度梯度对齐，我们未发现这是多任务学习独有问题。我们指出梯度幅值差异是主要区分因素。最后，我们比较了多任务学习与单任务学习在常见图像损坏上的特征迁移能力，未发现多任务学习能带来更优迁移能力的决定性证据。总体而言，我们发现单任务学习与多任务学习存在惊人相似性，建议在更广泛背景下综合考量两个领域的方法。