Interval Markov Decision Processes (IMDPs) are finite-state uncertain Markov models, where the transition probabilities belong to intervals. Recently, there has been a surge of research on employing IMDPs as abstractions of stochastic systems for control synthesis. However, due to the absence of algorithms for synthesis over IMDPs with continuous action-spaces, the action-space is assumed discrete a-priori, which is a restrictive assumption for many applications. Motivated by this, we introduce continuous-action IMDPs (caIMDPs), where the bounds on transition probabilities are functions of the action variables, and study value iteration for maximizing expected cumulative rewards. Specifically, we decompose the max-min problem associated to value iteration to $|\mathcal{Q}|$ max problems, where $|\mathcal{Q}|$ is the number of states of the caIMDP. Then, exploiting the simple form of these max problems, we identify cases where value iteration over caIMDPs can be solved efficiently (e.g., with linear or convex programming). We also gain other interesting insights: e.g., in certain cases where the action set $\mathcal{A}$ is a polytope, synthesis over a discrete-action IMDP, where the actions are the vertices of $\mathcal{A}$, is sufficient for optimality. We demonstrate our results on a numerical example. Finally, we include a short discussion on employing caIMDPs as abstractions for control synthesis.

翻译：区间马尔可夫决策过程（IMDPs）是具有有限状态的不确定马尔可夫模型，其转移概率属于区间。近期，利用IMDPs作为随机系统抽象以进行控制综合的研究激增。然而，由于缺乏针对连续动作空间IMDPs的综合算法，动作空间被先验地假定为离散的，这对许多应用而言是一种限制性假设。受此启发，我们引入了连续动作IMDPs（caIMDPs），其中转移概率的边界是动作变量的函数，并研究了用于最大化期望累积奖励的值迭代方法。具体而言，我们将值迭代相关的最大-最小问题分解为$|\mathcal{Q}|$个最大问题，其中$|\mathcal{Q}|$是caIMDP的状态数。然后，利用这些最大问题的简单形式，我们确定了caIMDPs上的值迭代能够高效求解（例如，通过线性或凸规划）的情形。我们还获得了其他有趣的见解：例如，在某些情况下，当动作集$\mathcal{A}$是多面体时，基于离散动作IMDP（其动作为$\mathcal{A}$的顶点）的综合足以达到最优性。我们通过数值示例展示了结果。最后，我们简要讨论了将caIMDPs用作控制综合的抽象。