Prior study has shown that pretrained language models (PLM) can boost the performance of text-based recommendation. In contrast to previous works that either use PLM to encode user history as a whole input text, or impose an additional aggregation network to fuse multi-turn history representations, we propose a unified local- and global-attention Transformer encoder to better model two-level contexts of user history. Moreover, conditioned on user history encoded by Transformer encoders, our framework leverages Transformer decoders to estimate the language perplexity of candidate text items, which can serve as a straightforward yet significant contrastive signal for user-item text matching. Based on this, our framework, UniTRec, unifies the contrastive objectives of discriminative matching scores and candidate text perplexity to jointly enhance text-based recommendation. Extensive evaluation shows that UniTRec delivers SOTA performance on three text-based recommendation tasks. Code is available at https://github.com/Veason-silverbullet/UniTRec.

翻译：摘要：先前研究表明，预训练语言模型能够提升基于文本的推荐性能。不同于以往工作仅使用预训练语言模型将用户历史编码为整体输入文本，或引入额外聚合网络融合多轮历史表征，我们提出一种统一的局部-全局注意力Transformer编码器，以更优地建模用户历史的两级上下文。此外，基于Transformer编码器编码的用户历史，本框架利用Transformer解码器估计候选文本项的语言困惑度，该困惑度可作为用户-文本匹配的直接且显著的对比信号。据此，本框架UniTRec统一了判别式匹配分数与候选文本困惑度的对比目标，以联合增强基于文本的推荐性能。广泛评估表明，UniTRec在三个基于文本的推荐任务中均达到最优性能。代码开源地址：https://github.com/Veason-silverbullet/UniTRec。