Ranking is a crucial module using in the recommender system. In particular, the ranking module using in our YoungTao recommendation scenario is to provide an ordered list of items to users, to maximize the click number throughout the recommendation session for each user. However, we found that the traditional ranking method for optimizing Click-Through rate(CTR) cannot address our ranking scenario well, since it completely ignores user leaving, and CTR is the optimization goal for the one-step recommendation. To effectively undertake the purpose of our ranking module, we propose a long-term optimization goal, named as CTE (Click-Through quantity expectation), for explicitly taking the behavior of user leaving into account. Based on CTE, we propose an effective model trained by reinforcement learning. Moreover, we build a simulation environment from offline log data for estimating PBR and CTR. We conduct extensive experiments on offline datasets and an online e-commerce scenario in TaoBao. Experimental results show that our method can boost performance effectively

翻译：摘要：排序是推荐系统中的关键模块。具体而言，我们YoungTao推荐场景中的排序模块旨在为用户提供有序的物品列表，以最大化每个用户在推荐会话中的点击次数。然而，我们发现传统以优化点击率（CTR）为目标的排序方法无法很好应对我们的排序场景，因为它完全忽略了用户离开行为，且CTR仅是一步推荐中的优化目标。为有效实现排序模块的既定目标，我们提出一种长期优化目标，命名为CTE（点击量期望），用于显式考虑用户离开行为。基于CTE，我们设计了一个通过强化学习训练的高效模型。此外，我们利用离线日志数据构建模拟环境以估算PBR和CTR。我们在离在线数据集以及淘宝在线电商场景中进行了大量实验，结果表明我们的方法能够有效提升性能。