Some recent \textit{news recommendation} (NR) methods introduce a Pre-trained Language Model (PLM) to encode news representation by following the vanilla pre-train and fine-tune paradigm with carefully-designed recommendation-specific neural networks and objective functions. Due to the inconsistent task objective with that of PLM, we argue that their modeling paradigm has not well exploited the abundant semantic information and linguistic knowledge embedded in the pre-training process. Recently, the pre-train, prompt, and predict paradigm, called \textit{prompt learning}, has achieved many successes in natural language processing domain. In this paper, we make the first trial of this new paradigm to develop a \textit{Prompt Learning for News Recommendation} (Prompt4NR) framework, which transforms the task of predicting whether a user would click a candidate news as a cloze-style mask-prediction task. Specifically, we design a series of prompt templates, including discrete, continuous, and hybrid templates, and construct their corresponding answer spaces to examine the proposed Prompt4NR framework. Furthermore, we use the prompt ensembling to integrate predictions from multiple prompt templates. Extensive experiments on the MIND dataset validate the effectiveness of our Prompt4NR with a set of new benchmark results.
翻译:近期一些新闻推荐方法采用预训练语言模型(PLM),遵循传统的预训练-微调范式,并配合精心设计的推荐专用神经网络与目标函数,对新闻表示进行编码。由于这些方法与PLM的任务目标存在不一致性,我们认为其建模范式并未充分利用预训练过程中蕴含的丰富语义信息与语言知识。近年来,预训练-提示-预测范式(即"提示学习")在自然语言处理领域取得了诸多成功。本文首次探索该范式,提出"面向新闻推荐的提示学习"(Prompt4NR)框架,将预测用户是否点击候选新闻的任务转化为完形填空风格的掩码预测任务。具体而言,我们设计了一系列提示模板(包括离散型、连续型及混合型模板),并构建对应的答案空间以验证所提出的Prompt4NR框架。此外,我们采用提示集成方法整合多个提示模板的预测结果。在MIND数据集上进行的大量实验验证了Prompt4NR的有效性,并取得了一系列新的基准结果。