This effort is focused on examining the behavior of reinforcement learning systems in personalization environments and detailing the differences in policy entropy associated with the type of learning algorithm utilized. We demonstrate that Policy Optimization agents often possess low-entropy policies during training, which in practice results in agents prioritizing certain actions and avoiding others. Conversely, we also show that Q-Learning agents are far less susceptible to such behavior and generally maintain high-entropy policies throughout training, which is often preferable in real-world applications. We provide a wide range of numerical experiments as well as theoretical justification to show that these differences in entropy are due to the type of learning being employed.

翻译：本工作聚焦于研究强化学习系统在个性化环境中的行为表现，并详细阐述与所用学习算法类型相关的策略熵差异。我们证明策略优化智能体在训练过程中常呈现低熵策略，这在实际应用中会导致智能体优先选择某些动作而回避其他动作。相反，我们还表明Q学习智能体对此类行为的敏感性远低于前者，且在训练过程中通常保持高熵策略，这在实际应用中往往更受青睐。通过大量数值实验与理论论证，我们证明这些熵值差异源于所采用的不同学习类型。