Dynamic Difficulty Adjustment (DDA) is a viable approach to enhance a player's experience in video games. Recently, Reinforcement Learning (RL) methods have been employed for DDA in non-competitive games; nevertheless, they rely solely on discrete state-action space with a small search space. In this paper, we propose a continuous RL-based DDA methodology for a visual working memory (VWM) game to handle the complex search space for the difficulty of memorization. The proposed RL-based DDA tailors game difficulty based on the player's score and game difficulty in the last trial. We defined a continuous metric for the difficulty of memorization. Then, we consider the task difficulty and the vector of difficulty-score as the RL's action and state, respectively. We evaluated the proposed method through a within-subject experiment involving 52 subjects. The proposed approach was compared with two rule-based difficulty adjustment methods in terms of player's score and game experience measured by a questionnaire. The proposed RL-based approach resulted in a significantly better game experience in terms of competence, tension, and negative and positive affect. Players also achieved higher scores and win rates. Furthermore, the proposed RL-based DDA led to a significantly less decline in the score in a 20-trial session.

翻译：动态难度调整（DDA）是提升玩家电子游戏体验的有效方法。近年来，强化学习（RL）方法已被应用于非竞技类游戏的动态难度调整，但现有方法仅依赖离散状态-动作空间且搜索空间较小。本文针对视觉工作记忆（VWM）游戏，提出了一种基于连续强化学习的动态难度调整方法，以处理记忆难度这一复杂搜索空间。所提出的RL-DDA方法基于玩家上一轮次的得分与游戏难度进行难度定制。我们定义了记忆难度的连续度量标准，并将任务难度与难度-得分向量分别作为RL的动作与状态。通过包含52名被试的受试者内实验，将所提方法与两种基于规则的难度调整方法进行了对比。实验通过问卷评估了玩家的得分与游戏体验。结果表明：在能力感、紧张感、正向情感与负向情感维度上，所提RL方法显著提升了游戏体验；玩家也获得了更高得分与胜率。此外，在20轮次实验中，所提RL-DDA方法使玩家得分的下降幅度显著降低。