Optimizing multiple objectives simultaneously is an important task for recommendation platforms to improve their performance. However, this task is particularly challenging since the relationships between different objectives are heterogeneous across different consumers and dynamically fluctuating according to different contexts. Especially in those cases when objectives become conflicting with each other, the result of recommendations will form a pareto-frontier, where the improvements of any objective comes at the cost of a performance decrease of another objective. Existing multi-objective recommender systems do not systematically consider such dynamic relationships; instead, they balance between these objectives in a static and uniform manner, resulting in only suboptimal multi-objective recommendation performance. In this paper, we propose a Deep Pareto Reinforcement Learning (DeepPRL) approach, where we (1) comprehensively model the complex relationships between multiple objectives in recommendations; (2) effectively capture personalized and contextual consumer preference for each objective to provide better recommendations; (3) optimize both the short-term and the long-term performance of multi-objective recommendations. As a result, our method achieves significant pareto-dominance over the state-of-the-art baselines in the offline experiments. Furthermore, we conducted a controlled experiment at the video streaming platform of Alibaba, where our method simultaneously improved three conflicting business objectives over the latest production system significantly, demonstrating its tangible economic impact in practice.

翻译：同时优化多个目标是推荐平台提升性能的重要任务。然而，由于不同目标之间的关系在不同消费者间具有异质性，且会随不同情境动态波动，该任务尤为困难。特别是在目标之间相互冲突的情况下，推荐结果将形成帕累托前沿，其中任一目标的改进均以另一目标性能下降为代价。现有的多目标推荐系统未能系统性地考虑此类动态关系；相反，它们以静态且统一的方式在这些目标间进行权衡，导致仅能获得次优的多目标推荐性能。本文提出一种深度帕累托强化学习（DeepPRL）方法，其中我们（1）全面建模推荐中多目标间的复杂关系；（2）有效捕捉每个目标的个性化及情境化消费者偏好以提供更优推荐；（3）优化多目标推荐的短期与长期性能。因此，我们的方法在离线实验中相较于现有先进基线实现了显著的帕累托支配。此外，我们在阿里巴巴的视频流媒体平台上进行了对照实验，结果表明该方法相较于最新生产系统同时显著提升了三个相互冲突的业务目标，证明了其在实践中切实的经济影响。