Reinforcement learning (RL) has gained popularity in the realm of recommender systems due to its ability to optimize long-term rewards and guide users in discovering relevant content. However, the successful implementation of RL in recommender systems is challenging because of several factors, including the limited availability of online data for training on-policy methods. This scarcity requires expensive human interaction for online model training. Furthermore, the development of effective evaluation frameworks that accurately reflect the quality of models remains a fundamental challenge in recommender systems. To address these challenges, we propose a comprehensive framework for synthetic environments that simulate human behavior by harnessing the capabilities of large language models (LLMs). We complement our framework with in-depth ablation studies and demonstrate its effectiveness with experiments on movie and book recommendations. By utilizing LLMs as synthetic users, this work introduces a modular and novel framework for training RL-based recommender systems. The software, including the RL environment, is publicly available.

翻译：强化学习因其能够优化长期奖励并引导用户发现相关内容，在推荐系统领域日益受到关注。然而，由于多种因素，在推荐系统中成功实施强化学习仍面临挑战，其中包括在线数据有限导致同策略方法训练困难。这种数据稀缺性要求通过昂贵的人工交互进行在线模型训练。此外，开发能准确反映模型质量的有效评估框架，仍是推荐系统领域的基础性挑战。为应对这些挑战，我们提出了一个利用大语言模型能力模拟人类行为的合成环境综合框架。我们通过深入的消融研究完善该框架，并在电影和图书推荐实验中验证其有效性。通过将大语言模型作为合成用户，本研究为训练基于强化学习的推荐系统引入了一个模块化的创新框架。相关软件（包括强化学习环境）已公开提供。