Sequential recommendation aims to model dynamic user behavior from historical interactions. Existing methods rely on either explicit item IDs or general textual features for sequence modeling to understand user preferences. While promising, these approaches still struggle to model cold-start items or transfer knowledge to new datasets. In this paper, we propose to model user preferences and item features as language representations that can be generalized to new items and datasets. To this end, we present a novel framework, named Recformer, which effectively learns language representations for sequential recommendation. Specifically, we propose to formulate an item as a "sentence" (word sequence) by flattening item key-value attributes described by text so that an item sequence for a user becomes a sequence of sentences. For recommendation, Recformer is trained to understand the "sentence" sequence and retrieve the next "sentence". To encode item sequences, we design a bi-directional Transformer similar to the model Longformer but with different embedding layers for sequential recommendation. For effective representation learning, we propose novel pretraining and finetuning methods which combine language understanding and recommendation tasks. Therefore, Recformer can effectively recommend the next item based on language representations. Extensive experiments conducted on six datasets demonstrate the effectiveness of Recformer for sequential recommendation, especially in low-resource and cold-start settings.

翻译：序列推荐旨在从历史交互中建模用户动态行为。现有方法依赖于显式项目ID或通用文本特征进行序列建模以理解用户偏好。尽管这些方法展现出潜力，但在处理冷启动项目或跨数据集迁移知识时仍面临困难。本文提出将用户偏好和项目特征建模为可泛化至新项目和新数据集的语言表征。为此，我们提出名为Recformer的新框架，该框架能有效学习序列推荐的语言表征。具体而言，我们通过将文本描述的键值属性扁平化，将项目形式化为"句子"（词序列），使得用户行为序列转化为句子序列。在推荐任务中，Recformer通过训练理解"句子"序列并预测下一"句子"。为编码项目序列，我们设计了一种双向Transformer结构，该结构基于Longformer框架并针对序列推荐定制了嵌入层。通过融合语言理解与推荐任务，我们提出创新的预训练与微调方法以实现高效表征学习。因此，Recformer能基于语言表征有效推荐下一项目。在六个数据集上的大量实验证明Recformer在序列推荐（尤其在低资源与冷启动场景）中的有效性。