Traditional recommender systems such as matrix factorization methods rely on learning a shared dense embedding space to represent both items and user preferences. Sequence models such as RNN, GRUs, and, recently, Transformers have also excelled in the task of sequential recommendation. This task requires understanding the sequential structure present in users' historical interactions to predict the next item they may like. Building upon the success of Large Language Models (LLMs) in a variety of tasks, researchers have recently explored using LLMs that are pretrained on vast corpora of text for sequential recommendation. To use LLMs in sequential recommendations, both the history of user interactions and the model's prediction of the next item are expressed in text form. We propose CALRec, a two-stage LLM finetuning framework that finetunes a pretrained LLM in a two-tower fashion using a mixture of two contrastive losses and a language modeling loss: the LLM is first finetuned on a data mixture from multiple domains followed by another round of target domain finetuning. Our model significantly outperforms many state-of-the-art baselines (+37% in Recall@1 and +24% in NDCG@10) and systematic ablation studies reveal that (i) both stages of finetuning are crucial, and, when combined, we achieve improved performance, and (ii) contrastive alignment is effective among the target domains explored in our experiments.

翻译：传统推荐系统（如矩阵分解方法）依赖于学习共享的密集嵌入空间来表示物品和用户偏好。序列模型（如RNN、GRU以及近期的Transformer）在序列推荐任务中同样表现出色，该任务需理解用户历史交互中的序列结构以预测其可能喜欢的下一个物品。基于大语言模型（LLM）在多种任务中的成功，研究者近期探索了利用预训练于大规模文本语料库的LLM进行序列推荐。为在序列推荐中使用LLM，用户交互历史与模型对下一个物品的预测均以文本形式表达。我们提出CALRec——一种两阶段LLM微调框架，通过双塔结构混合使用两种对比损失和语言建模损失对预训练LLM进行微调：首先在多领域数据混合上进行微调，随后进行目标领域微调。我们的模型显著优于多项最先进基线（Recall@1提升37%，NDCG@10提升24%），系统的消融实验揭示：（i）两阶段微调均至关重要，且组合使用时性能更优；（ii）在我们实验探索的目标领域中，对比对齐方法具有有效性。