Many sequential decision-making problems need optimization of different objectives which possibly conflict with each other. The conventional way to deal with a multi-task problem is to establish a scalar objective function based on a linear combination of different objectives. However, for the case of having conflicting objectives with different scales, this method needs a trial-and-error approach to properly find proper weights for the combination. As such, in most cases, this approach cannot guarantee an optimal Pareto solution. In this paper, we develop a single-agent scale-independent multi-objective reinforcement learning on the basis of the Advantage Actor-Critic (A2C) algorithm. A convergence analysis is then done for the devised multi-objective algorithm providing a convergence-in-mean guarantee. We then perform some experiments over a multi-task problem to evaluate the performance of the proposed algorithm. Simulation results show the superiority of developed multi-objective A2C approach against the single-objective algorithm.

翻译：许多序贯决策问题需要优化可能相互冲突的不同目标。处理多任务问题的传统方法是基于不同目标的线性组合构建标量目标函数。然而，当存在尺度不同的冲突目标时，该方法需要通过试错法来恰当确定组合权重。因此，在大多数情况下，这种方法无法保证得到帕累托最优解。本文基于优势行动者-评论家（A2C）算法，提出了一种单智能体尺度无关的多目标强化学习方法。随后对设计的算法进行了收敛性分析，保证了均值收敛性。我们进一步在多任务问题上开展实验以评估所提算法的性能。仿真结果表明，本文提出的多目标A2C方法相较于单目标算法具有优越性。