We consider Markov decision processes where the state of the chain is only given at chosen observation times and of a cost. Optimal strategies involve the optimisation of observation times as well as the subsequent action values. We consider the finite horizon and discounted infinite horizon problems, as well as an extension with parameter uncertainty. By including the time elapsed from observations as part of the augmented Markov system, the value function satisfies a system of quasi-variational inequalities (QVIs). Such a class of QVIs can be seen as an extension to the interconnected obstacle problem. We prove a comparison principle for this class of QVIs, which implies uniqueness of solutions to our proposed problem. Penalty methods are then utilised to obtain arbitrarily accurate solutions. Finally, we perform numerical experiments on three applications which illustrate our framework.

翻译：本文研究一类马尔可夫决策过程，其中链的状态仅在选定的观测时刻以一定成本获取。最优策略涉及观测时刻的优化以及后续行动值的选取。我们考虑有限时域和折扣无限时域问题，并拓展至参数不确定性情形。通过将观测间隔时间作为增广马尔可夫系统的一部分，价值函数满足一类拟变分不等式系统（QVIs）。这类QVIs可视为互联障碍问题的扩展。我们证明了该类QVIs的比较原理，进而确保所提问题解的唯一性。采用惩罚方法可获取任意精度的近似解。最后，通过三个应用场景的数值实验验证了所提框架的有效性。