This paper analyzes finite state Markov Decision Processes (MDPs) with uncertain parameters in compact sets and re-examines results from robust MDP via set-based fixed point theory. To this end, we generalize the Bellman and policy evaluation operators to contracting operators on the value function space and denote them as \emph{value operators}. We lift these value operators to act on \emph{sets} of value functions and denote them as \emph{set-based value operators}. We prove that the set-based value operators are \emph{contractions} in the space of compact value function sets. Leveraging insights from set theory, we generalize the rectangularity condition in classic robust MDP literature to a containment condition for all value operators, which is weaker and can be applied to a larger set of parameter-uncertain MDPs and contracting operators in dynamic programming. We prove that both the rectangularity condition and the containment condition sufficiently ensure that the set-based value operator's fixed point set contains its own extrema elements. For convex and compact sets of uncertain MDP parameters, we show equivalence between the classic robust value function and the supremum of the fixed point set of the set-based Bellman operator. Under dynamically changing MDP parameters in compact sets, we prove a set convergence result for value iteration, which otherwise may not converge to a single value function. Finally, we derive novel guarantees for probabilistic path-planning problems in planet exploration and stratospheric station-keeping.

翻译：本文分析了参数位于紧集内的有限状态马尔可夫决策过程（MDP），并借助基于集合的不动点理论重新审视了鲁棒MDP的相关结论。为此，我们将贝尔曼算子和策略评估算子推广为值函数空间上的压缩算子，并称之为**值算子**。我们将这些值算子提升至作用于值函数的**集合**上，并称之为**基于集合的值算子**。我们证明，在紧值函数集合空间中，基于集合的值算子是**压缩映射**。借鉴集合论的思想，我们将经典鲁棒MDP文献中的矩形条件推广为所有值算子的包含条件，该条件更弱，可应用于更大一类参数不确定的MDP以及动态规划中的压缩算子。我们证明，矩形条件和包含条件均足以确保基于集合的值算子的不动点集包含其自身的极值元素。对于凸且紧的参数不确定MDP集合，我们证明经典鲁棒值函数与基于集合的贝尔曼算子的不动点集的上确界等价。在紧集内动态变化的MDP参数下，我们证明了值迭代的集合收敛性结果，而值迭代在通常情况下可能不收敛于单一值函数。最后，我们为行星探索和平流层位置保持中的概率路径规划问题推导了新颖的保证。