Sequential Recommendation is a popular recommendation task that uses the order of user-item interaction to model evolving users' interests and sequential patterns in their behaviour. Current state-of-the-art Transformer-based models for sequential recommendation, such as BERT4Rec and SASRec, generate sequence embeddings and compute scores for catalogue items, but the increasing catalogue size makes training these models costly. The Joint Product Quantisation (JPQ) method, originally proposed for passage retrieval, markedly reduces the size of the retrieval index with minimal effect on model effectiveness, by replacing passage embeddings with a limited number of shared sub-embeddings. This paper introduces RecJPQ, a novel adaptation of JPQ for sequential recommendations, which takes the place of item embeddings tensor and replaces item embeddings with a concatenation of a limited number of shared sub-embeddings and, therefore, limits the number of learnable model parameters. The main idea of RecJPQ is to split items into sub-item entities before training the main recommendation model, which is inspired by splitting words into tokens and training tokenisers in language models. We apply RecJPQ to SASRec, BERT4Rec, and GRU4rec models on three large-scale sequential datasets. Our results showed that RecJPQ could notably reduce the model size (e.g., 48% reduction for the Gowalla dataset with no effectiveness degradation). RecJPQ can also improve model performance through a regularisation effect (e.g. +0.96% NDCG@10 improvement on the Booking.com dataset). Overall, RecJPQ allows the training of state-of-the-art transformer recommenders in industrial applications, where datasets with millions of items are common.

翻译：顺序推荐是一种流行的推荐任务，通过用户-物品交互的顺序来建模用户兴趣的演变及其行为中的序列模式。当前基于Transformer的顺序推荐模型（如BERT4Rec和SASRec）虽能生成序列嵌入并计算目录物品的分数，但日益增长的目录规模使模型训练成本高昂。最初为篇章检索提出的联合乘积量化（JPQ）方法，通过用有限数量的共享子嵌入替换篇章嵌入，在不显著影响模型效果的前提下大幅缩小检索索引规模。本文提出RecJPQ，这是对JPQ在顺序推荐场景中的创新性适配，它取代物品嵌入张量，通过拼接有限数量的共享子嵌入来替代物品嵌入，从而限制可学习模型参数的数量。RecJPQ的核心思想是在训练主推荐模型之前将物品拆分为子物品实体，这一灵感来源于语言模型中分词为令牌并训练分词器的方法。我们将RecJPQ应用于SASRec、BERT4Rec和GRU4Rec三个大规模顺序数据集上的模型。结果表明，RecJPQ能显著减小模型体积（例如在Gowalla数据集上减少48%且无效果损失），还能通过正则化效应提升模型性能（如在Booking.com数据集上NDCG@10提升0.96%）。总体而言，RecJPQ使得在工业应用中训练拥有数百万物品数据集的当前最优Transformer推荐模型成为可能。