An ultimate goal of recommender systems (RS) is to improve user engagement. Reinforcement learning (RL) is a promising paradigm for this goal, as it directly optimizes overall performance of sequential recommendation. However, many existing RL-based approaches induce huge computational overhead, because they require not only the recommended items but also all other candidate items to be stored. This paper proposes an efficient alternative that does not require the candidate items. The idea is to model the correlation between user engagement and items directly from data. Moreover, the proposed approach consider randomness in user feedback and termination behavior, which are ubiquitous for RS but rarely discussed in RL-based prior work. With online A/B experiments on real-world RS, we confirm the efficacy of the proposed approach and the importance of modeling the two types of randomness.

翻译：推荐系统（RS）的终极目标是提升用户参与度。强化学习（RL）作为实现该目标的有效范式，能够直接优化序列推荐的总体性能。然而，现有基于RL的方法通常需要同时存储推荐物品与所有候选物品，导致巨大的计算开销。本文提出一种无需候选物品的高效替代方案，其核心思想是从数据中直接建模用户参与度与物品之间的关联。此外，该方法考虑了用户反馈与终止行为中的随机性——这两种随机性在RS中普遍存在，但在以往基于RL的研究中鲜有讨论。通过真实RS的在线A/B实验，我们验证了所提方法的有效性，并证实了建模这两种随机性的关键作用。