This paper studies the computation of robust deterministic policies for Markov Decision Processes (MDPs) in the Lightning Does Not Strike Twice (LDST) model of Mannor, Mebel and Xu (ICML '12). In this model, designed to provide robustness in the face of uncertain input data while not being overly conservative, transition probabilities and rewards are uncertain and the uncertainty set is constrained by a budget that limits the number of states whose parameters can deviate from their nominal values. Mannor et al. (ICML '12) showed that optimal randomized policies for MDPs in the LDST regime can be efficiently computed when only the rewards are affected by uncertainty. In contrast to these findings, we observe that the computation of optimal deterministic policies is $N\!P$-hard even when only a single terminal reward may deviate from its nominal value and the MDP consists of $2$ time periods. For this hard special case, we then derive a constant-factor approximation algorithm by combining two relaxations based on the Knapsack Cover and Generalized Assignment problem, respectively. For the general problem with possibly a large number of deviations and a longer time horizon, we derive strong inapproximability results for computing robust deterministic policies as well as $\Sigma_2^p$-hardness, indicating that the general problem does not even admit a compact mixed integer programming formulation.

翻译：本文研究了在Mannor、Mebel和Xu（ICML '12）提出的"闪电不会两次击中同一处"（LDST）模型中，为马尔可夫决策过程（MDPs）计算鲁棒确定性策略的问题。该模型旨在为不确定输入数据提供鲁棒性，同时避免过于保守，其中转移概率和奖励是不确定的，且不确定性集合受限于一个预算，该预算限制了参数可能偏离其标称值的状态数量。Mannor等人（ICML '12）表明，当仅奖励受不确定性影响时，LDST机制下MDPs的最优随机策略可以被高效计算。与这些发现相反，我们观察到，即使仅有一个终端奖励可能偏离其标称值且MDP仅包含$2$个时间周期，计算最优确定性策略也是$N\!P$-难的。针对这一困难的特殊情况，我们通过结合分别基于背包覆盖问题和广义分配问题的两种松弛方法，推导出一种常数因子近似算法。对于可能包含大量偏差和更长时域的一般问题，我们推导出计算鲁棒确定性策略的强不可近似性结果以及$\Sigma_2^p$-难度，表明该一般问题甚至不存在紧凑的混合整数规划公式。