A central task in control theory, artificial intelligence, and formal methods is to synthesize reward-maximizing strategies for agents that operate in partially unknown environments. In environments modeled by gray-box Markov decision processes (MDPs), the impact of the agents' actions are known in terms of successor states but not the stochastics involved. In this paper, we devise a strategy synthesis algorithm for gray-box MDPs via reinforcement learning that utilizes interval MDPs as internal model. To compete with limited sampling access in reinforcement learning, we incorporate two novel concepts into our algorithm, focusing on rapid and successful learning rather than on stochastic guarantees and optimality: lower confidence bound exploration reinforces variants of already learned practical strategies and action scoping reduces the learning action space to promising actions. We illustrate benefits of our algorithms by means of a prototypical implementation applied on examples from the AI and formal methods communities.

翻译：控制理论、人工智能和形式化方法中的一项核心任务是为在部分未知环境中运行的智能体综合奖励最大化策略。在以灰箱马尔可夫决策过程（MDP）建模的环境中，智能体动作对后继状态的影响是已知的，但涉及的随机性未知。本文通过强化学习提出了一种针对灰箱MDP的策略综合算法，该算法采用区间MDP作为内部模型。为应对强化学习中受限的采样访问，我们在算法中融入两个新颖概念，聚焦于快速且成功的策略学习，而非追求随机性保证与最优性：下置信界探索强化已学到的实用策略变体，而动作范围缩减则将学习动作空间限制在具有潜力的动作上。我们通过在AI与形式化方法领域的实例上应用原型实现，展示了该算法的优势。