While personalization increases the utility of recommender systems, it also brings the issue of filter bubbles. E.g., if the system keeps exposing and recommending the items that the user is interested in, it may also make the user feel bored and less satisfied. Existing work studies filter bubbles in static recommendation, where the effect of overexposure is hard to capture. In contrast, we believe it is more meaningful to study the issue in interactive recommendation and optimize long-term user satisfaction. Nevertheless, it is unrealistic to train the model online due to the high cost. As such, we have to leverage offline training data and disentangle the causal effect on user satisfaction. To achieve this goal, we propose a counterfactual interactive recommender system (CIRS) that augments offline reinforcement learning (offline RL) with causal inference. The basic idea is to first learn a causal user model on historical data to capture the overexposure effect of items on user satisfaction. It then uses the learned causal user model to help the planning of the RL policy. To conduct evaluation offline, we innovatively create an authentic RL environment (KuaiEnv) based on a real-world fully observed user rating dataset. The experiments show the effectiveness of CIRS in bursting filter bubbles and achieving long-term success in interactive recommendation. The implementation of CIRS is available via https://github.com/chongminggao/CIRS-codes.

翻译：尽管个性化提升了推荐系统的效用，但也带来了过滤气泡问题。例如，若系统持续曝光和推荐用户感兴趣的项目，可能导致用户感到厌倦并降低满意度。现有研究主要针对静态推荐中的过滤气泡，但过度曝光的效应在此类场景中难以捕捉。相比之下，我们认为在交互式推荐中研究该问题并优化长期用户满意度更具实际意义。然而，由于在线训练模型成本高昂，难以在实践中落地。为此，我们必须利用离线训练数据，并解耦用户满意度中的因果效应。为实现这一目标，我们提出了一种反事实交互式推荐系统（CIRS），它将离线强化学习（offline RL）与因果推断相结合。其核心思想是：首先基于历史数据学习一个因果用户模型，以捕捉项目过度曝光对用户满意度的影响；随后利用该因果用户模型辅助强化学习策略的规划。为了进行离线评估，我们创新性地基于真实世界全观测用户评分数据集构建了一个真实的强化学习环境（KuaiEnv）。实验表明，CIRS在打破过滤气泡和实现交互式推荐长期成功方面具有有效性。CIRS的实现代码可通过https://github.com/chongminggao/CIRS-codes获取。