Multi-objective optimization (MOO) has become an influential framework in many machine learning problems with multiple objectives such as learning with multiple criteria and multi-task learning (MTL). In this paper, we propose a new direction-oriented multi-objective problem by regularizing the common descent direction within a neighborhood of a direction that optimizes a linear combination of objectives such as the average loss in MTL. This formulation includes GD and MGDA as special cases, enjoys the direction-oriented benefit as in CAGrad, and facilitates the design of stochastic algorithms. To solve this problem, we propose Stochastic Direction-oriented Multi-objective Gradient descent (SDMGrad) with simple SGD type of updates, and its variant SDMGrad-OS with an efficient objective sampling in the setting where the number of objectives is large. For a constant-level regularization parameter $\lambda$, we show that SDMGrad and SDMGrad-OS provably converge to a Pareto stationary point with improved complexities and milder assumptions. For an increasing $\lambda$, this convergent point reduces to a stationary point of the linear combination of objectives. We demonstrate the superior performance of the proposed methods in a series of tasks on multi-task supervised learning and reinforcement learning. Code is provided at https://github.com/ml-opt-lab/sdmgrad.

翻译：多目标优化（MOO）已成为许多具有多个目标的机器学习问题中的关键框架，例如多准则学习和多任务学习（MTL）。本文通过正则化通用下降方向，使其接近优化多目标线性组合（如MTL中的平均损失）的方向邻域，提出了一种新的面向方向的多目标问题。该表述将GD和MGDA作为特例纳入，兼具CAGrad等方法的面向方向优势，并便于随机算法的设计。为解决此问题，我们提出了具有简单SGD类型更新的随机面向方向多目标梯度下降（SDMGrad），及其在目标数量较大时采用高效目标采样的变体SDMGrad-OS。对于常数级正则化参数$\lambda$，我们证明SDMGrad和SDMGrad-OS能可证明地收敛到帕累托平稳点，且具有更优的复杂度和更弱的假设。当$\lambda$递增时，该收敛点退化为多目标线性组合的平稳点。我们在一系列多任务监督学习和强化学习任务中展示了所提方法的优越性能。代码提供于https://github.com/ml-opt-lab/sdmgrad。