Optimizing multiple objectives simultaneously is an important task for recommendation platforms to improve their performance. However, this task is particularly challenging since the relationships between different objectives are heterogeneous across different consumers and dynamically fluctuating according to different contexts. Especially in those cases when objectives become conflicting with each other, the result of recommendations will form a pareto-frontier, where the improvements of any objective comes at the cost of a performance decrease of another objective. Existing multi-objective recommender systems do not systematically consider such dynamic relationships; instead, they balance between these objectives in a static and uniform manner, resulting in only suboptimal multi-objective recommendation performance. In this paper, we propose a Deep Pareto Reinforcement Learning (DeepPRL) approach, where we (1) comprehensively model the complex relationships between multiple objectives in recommendations; (2) effectively capture personalized and contextual consumer preference for each objective to provide better recommendations; (3) optimize both the short-term and the long-term performance of multi-objective recommendations. As a result, our method achieves significant pareto-dominance over the state-of-the-art baselines in the offline experiments. Furthermore, we conducted a controlled experiment at the video streaming platform of Alibaba, where our method simultaneously improved three conflicting business objectives over the latest production system significantly, demonstrating its tangible economic impact in practice.

翻译：同时优化多个目标是推荐平台提升性能的重要任务。然而，由于不同目标之间的关系在不同消费者间具有异质性，且会随不同情境动态波动，该任务尤为困难。尤其在目标间相互冲突的情况下，推荐结果将形成帕累托前沿，其中任一目标的改进均以另一目标性能下降为代价。现有的多目标推荐系统未能系统性地考虑此类动态关系，而是以静态、统一的方式平衡这些目标，导致多目标推荐性能仅达到次优水平。本文提出一种深度帕累托强化学习（DeepPRL）方法，其中我们（1）全面建模推荐中多目标间的复杂关系；（2）有效捕捉每个目标的个性化与情境化消费者偏好以提供更优推荐；（3）同时优化多目标推荐的短期与长期性能。实验结果表明，在离线实验中，我们的方法相较于现有先进基线模型实现了显著的帕累托支配性。此外，我们在阿里巴巴视频流媒体平台上进行了受控实验，结果显示本方法在最新生产系统基础上，同步显著提升了三个相互冲突的商业目标，证明了其在实际应用中的切实经济影响。