To drive purchase in online advertising, it is of the advertiser's great interest to optimize the sequential advertising strategy whose performance and interpretability are both important. The lack of interpretability in existing deep reinforcement learning methods makes it not easy to understand, diagnose and further optimize the strategy. In this paper, we propose our Deep Intents Sequential Advertising (DISA) method to address these issues. The key part of interpretability is to understand a consumer's purchase intent which is, however, unobservable (called hidden states). In this paper, we model this intention as a latent variable and formulate the problem as a Partially Observable Markov Decision Process (POMDP) where the underlying intents are inferred based on the observable behaviors. Large-scale industrial offline and online experiments demonstrate our method's superior performance over several baselines. The inferred hidden states are analyzed, and the results prove the rationality of our inference.

翻译：为了推动在线广告的购买,广告商非常希望优化连续的广告战略,其性能和解释都很重要。现有深层强化学习方法缺乏解释性,因此难以理解、诊断和进一步优化战略。在本文中,我们提出了我们深层元素序列广告(DISA)解决这些问题的方法。解释性的关键部分是理解消费者购买意图,然而,这种意图是不可观察到的(所谓的隐蔽状态 ) 。在本文中,我们将这一意图作为潜在变量,并将问题表述为部分可观测的Markov 决策程序(POMDP ), 其基本意图根据可观察的行为推断。大规模工业离线和在线实验显示了我们的方法在几个基线上的优异性。对隐蔽状态进行了分析,结果证明了我们判断的合理性。