Many sequential decision-making problems need optimization of different objectives which possibly conflict with each other. The conventional way to deal with a multi-task problem is to establish a scalar objective function based on a linear combination of different objectives. However, for the case of having conflicting objectives with different scales, this method needs a trial-and-error approach to properly find proper weights for the combination. As such, in most cases, this approach cannot guarantee an optimal Pareto solution. In this paper, we develop a single-agent scale-independent multi-objective reinforcement learning on the basis of the Advantage Actor-Critic (A2C) algorithm. A convergence analysis is then done for the devised multi-objective algorithm providing a convergence-in-mean guarantee. We then perform some experiments over a multi-task problem to evaluate the performance of the proposed algorithm. Simulation results show the superiority of developed multi-objective A2C approach against the single-objective algorithm.

翻译：许多序贯决策问题需要优化可能相互冲突的不同目标。传统处理多任务问题的方法是基于不同目标的线性组合构建标量目标函数。然而，当存在尺度不同的冲突目标时，该方法需要通过试错法才能适当确定组合权重。因此，在大多数情况下，这种方法无法保证获得最优帕累托解。本文基于优势行动者-评论家(A2C)算法，提出了一种单智能体尺度无关的多目标强化学习方法。随后，对所设计的多目标算法进行收敛性分析，给出了均值收敛的保证。我们进一步在多任务问题上进行实验以评估所提算法的性能。仿真结果表明，所开发的多目标A2C方法优于单目标算法。