Multi-task learning (MTL) algorithms typically rely on schemes that combine different task losses or their gradients through weighted averaging. These methods aim to find Pareto stationary points by using heuristics that require access to task loss values, gradients, or both. In doing so, a central challenge arises because task losses can be arbitrarily scaled relative to one another, causing certain tasks to dominate training and degrade overall performance. A recent advance in cooperative bargaining theory, the Direction-based Bargaining Solution (DiBS), yields Pareto stationary solutions immune to task domination because of its invariance to monotonic nonaffine task loss transformations. However, the convergence behavior of DiBS in nonconvex MTL settings is currently not understood. To this end, we prove that under standard assumptions, a subsequence of DiBS iterates converges to a Pareto stationary point when task losses are nonconvex, and propose DiBS-MTL, an adaptation of DiBS to the MTL setting which is more computationally efficient that prior bargaining-inspired MTL approaches. Finally, we empirically show that DiBS-MTL is competitive with leading MTL methods on standard benchmarks, and it drastically outperforms state-of-the-art baselines in multiple examples with poorly-scaled task losses, highlighting the importance of invariance to nonaffine monotonic transformations of the loss landscape. Code available at https://github.com/suryakmurthy/dibs-mtl

翻译：多任务学习（MTL）算法通常依赖于通过加权平均组合不同任务损失或其梯度的方案。这些方法旨在通过使用需要访问任务损失值、梯度或两者的启发式方法，找到帕累托平稳点。在此过程中，一个核心挑战出现，因为任务损失可以任意地相互缩放，导致某些任务主导训练并降低整体性能。合作博弈理论的最新进展——基于方向的讨价还价解（DiBS），因其对单调非仿射任务损失变换的不变性，产生了不受任务主导影响的帕累托平稳解。然而，DiBS在非凸MTL设置中的收敛行为目前尚未被理解。为此，我们证明在标准假设下，当任务损失非凸时，DiBS迭代的子序列收敛到一个帕累托平稳点，并提出了DiBS-MTL，这是DiBS在MTL设置中的一种适应性改进，其计算效率高于先前受博弈启发的MTL方法。最后，我们通过实验表明，DiBS-MTL在标准基准测试中与领先的MTL方法具有竞争力，并且在多个任务损失缩放不当的示例中显著优于最先进的基线方法，突显了损失函数非仿射单调变换不变性的重要性。代码可在 https://github.com/suryakmurthy/dibs-mtl 获取。