We consider lexicographic bi-objective problems on Markov Decision Processes (MDPs), where we optimize one objective while guaranteeing optimality of another. We propose a two-stage technique for solving such problems when the objectives are related (in a way that we formalize). We instantiate our technique for two natural pairs of objectives: minimizing the (conditional) expected number of steps to a target while guaranteeing the optimal probability of reaching it; and maximizing the (conditional) expected average reward while guaranteeing an optimal probability of staying safe (w.r.t. some safe set of states). For the first combination of objectives, which covers the classical frozen lake environment from reinforcement learning, we also report on experiments performed using a prototype implementation of our algorithm and compare it with what can be obtained from state-of-the-art probabilistic model checkers solving optimal reachability.

翻译：我们考虑马尔可夫决策过程（MDP）中的字典序双目标问题，即在一个目标上保证最优性的同时优化另一个目标。我们提出了一种两阶段技术来解决当目标相关时（以我们形式化的方式）的此类问题。我们将此技术应用于两个自然的目标对：在确保到达目标点的最优概率的同时最小化（条件化）期望步数；以及在确保最优安全概率（相对于某个安全状态集）的同时最大化（条件化）期望平均奖励。针对第一个目标组合（涵盖了经典强化学习中的冰冻湖环境），我们还报告了使用算法原型实现进行的实验，并将其与最先进的概率模型检验器求解最优可达性所获得的结果进行了比较。