Sequential recommendations (SR) with transformer-based architectures are widely adopted in real-world applications, where SR models require frequent retraining to adapt to ever-changing user preferences. However, training transformer-based SR models often encounters a high computational cost associated with scoring extensive item catalogs, often exceeding thousands of items. This occurs mainly due to the use of cross-entropy loss, where peak memory scales proportionally to catalog size, batch size, and sequence length. Recognizing this, practitioners in the field of recommendation systems typically address memory consumption by integrating the cross-entropy (CE) loss with negative sampling, thereby reducing the explicit memory demands of the final layer. However, a small number of negative samples would degrade model performance, and as we demonstrate in our work, increasing the number of negative samples and the batch size further improves the model's performance, but rapidly starts to exceed industrial GPUs' size (~40Gb). In this work, we introduce the CCE- method, which offers a GPU-efficient implementation of the CE loss with negative sampling. Our method accelerates training by up to two times while reducing memory consumption by more than 10 times. Leveraging the memory savings afforded by using CCE- for model training, it becomes feasible to enhance its accuracy on datasets with a large item catalog compared to those trained with original PyTorch-implemented loss functions. Finally, we perform an analysis of key memory-related hyperparameters and highlight the necessity of a delicate balance among these factors. We demonstrate that scaling both the number of negative samples and batch size leads to better results rather than maximizing only one of them. To facilitate further adoption of CCE-, we release a Triton kernel that efficiently implements the proposed method.

翻译：基于Transformer架构的序列推荐（SR）模型在现实应用中广泛部署，这些模型需要频繁重训练以适应不断变化的用户偏好。然而，训练基于Transformer的SR模型时，对包含数千个项目的庞大目录进行评分往往面临高昂的计算成本。这主要源于交叉熵损失函数的使用：峰值内存与目录规模、批大小和序列长度呈正比缩放。认识到这一点，推荐系统领域从业者通常通过将交叉熵损失与负采样相结合来缓解内存消耗，从而降低最终层的显式内存需求。但少量负样本会损害模型性能，且如我们研究所示，增加负样本数量与批大小能进一步提升模型性能，却会迅速突破工业级GPU的内存上限（约40GB）。本研究提出CCE-方法，该方法为含负采样的交叉熵损失提供了GPU高效实现。我们的方法将训练速度提升至两倍，同时内存消耗降低超过十倍。利用CCE-训练模型节省的内存，相较于使用原始PyTorch实现的损失函数，我们能够在拥有大规模项目目录的数据集上提升模型精度。最后，我们分析关键内存相关超参数，强调需精细平衡这些因素。实验证明，同时扩展负样本数量与批大小比仅最大化单一参数能获得更优结果。为促进CCE-的推广应用，我们发布了实现所提方法的高效Triton内核。