Many games feature a progression of levels that doesn't adapt to the player. This can be problematic because some players may get stuck if the progression is too difficult, while others may find it boring if the progression is too slow to get to more challenging levels. This can be addressed by building levels based on the player's performance and preferences. In this work, we formulate the problem of generating levels for a player as a Markov Decision Process (MDP) and use adaptive dynamic programming (ADP) to solve the MDP before assembling a level. We tested with two case studies and found that using an ADP outperforms two baselines. Furthermore, we experimented with player proxies and switched them in the middle of play, and we show that a simple modification prior to running ADP results in quick adaptation. By using ADP, which searches the entire MDP, we produce a dynamic progression of levels that adapts to the player.

翻译：许多游戏中的关卡 progression 无法根据玩家进行调整。这可能引发问题：若 progression 难度过高，部分玩家可能卡关；若难度提升过慢，难以到达更具挑战性的关卡，玩家则可能感到枯燥。针对这一问题，可通过基于玩家表现与偏好构建关卡来解决。本研究将面向玩家的关卡生成问题建模为马尔可夫决策过程（MDP），并在组装关卡前采用自适应动态规划（ADP）求解该 MDP。通过两项案例研究，我们发现使用 ADP 的性能优于两种基线方法。此外，我们采用玩家代理进行实验，并在游戏过程中切换代理，结果表明：在运行 ADP 前进行简单修改即可实现快速适应。通过利用搜索整个 MDP 的 ADP，我们生成了能够动态适应玩家的关卡 progression。