Sequential recommendation aims to model dynamic user behavior from historical interactions. Existing methods rely on either explicit item IDs or general textual features for sequence modeling to understand user preferences. While promising, these approaches still struggle to model cold-start items or transfer knowledge to new datasets. In this paper, we propose to model user preferences and item features as language representations that can be generalized to new items and datasets. To this end, we present a novel framework, named Recformer, which effectively learns language representations for sequential recommendation. Specifically, we propose to formulate an item as a "sentence" (word sequence) by flattening item key-value attributes described by text so that an item sequence for a user becomes a sequence of sentences. For recommendation, Recformer is trained to understand the "sentence" sequence and retrieve the next "sentence". To encode item sequences, we design a bi-directional Transformer similar to the model Longformer but with different embedding layers for sequential recommendation. For effective representation learning, we propose novel pretraining and finetuning methods which combine language understanding and recommendation tasks. Therefore, Recformer can effectively recommend the next item based on language representations. Extensive experiments conducted on six datasets demonstrate the effectiveness of Recformer for sequential recommendation, especially in low-resource and cold-start settings.

翻译：序列推荐旨在从历史交互中建模动态用户行为。现有方法依赖显式物品ID或通用文本特征进行序列建模以理解用户偏好。尽管这些方法展现出潜力，但在处理冷启动物品或跨数据集迁移知识时仍存在困难。本文提出将用户偏好与物品特征建模为可泛化至新物品和数据集的**语言表示**。为此，我们提出一种名为Recformer的创新框架，该框架能高效学习序列推荐的语言表示。具体而言，我们通过展平以文本描述的物品键-值属性，将每个物品构建为"句子"（词序列），从而使用户的物品序列转化为句子序列。在推荐任务中，Recformer通过训练理解"句子"序列并检索下一个"句子"。为编码物品序列，我们设计了类似于Longformer的双向Transformer，但针对序列推荐采用不同的嵌入层。为实现高效表示学习，我们提出结合语言理解与推荐任务的新型预训练与微调方法。因此，Recformer能够基于语言表示有效推荐下一物品。在六个数据集上的广泛实验表明，Recformer在序列推荐任务中（尤其在低资源与冷启动场景下）具有显著有效性。