There has been a growing interest in benchmarking sequential recommendation models and reproducing/improving existing models. For example, Rendle et al. improved matrix factorization models by tuning their parameters and hyperparameters. Petrov and Macdonald developed a more efficient and effective implementation of BERT4Rec, which resolved inconsistencies in performance comparison between BERT4Rec and SASRec in previous works. In particular, BERT4Rec and SASRec share a similar network structure, with the main difference lying in their training objective/loss function. Therefore, we analyzed the advantages and disadvantages of commonly used loss functions in sequential recommendation and proposed an improved loss function that leverages their strengths. We conduct extensive experiments on two influential open-source libraries, and the results demonstrate that our improved loss function significantly enhances the performance of GRU4Rec, SASRec, SR-GNN, and S3Rec models, improving their benchmarks significantly. Furthermore, the improved SASRec benchmark outperforms BERT4Rec on the ML-1M and Beauty datasets and achieves similar results to BERT4Rec on the ML-20M and Steam datasets. We also reproduce the results of the BERT4Rec model on the Beauty dataset. Finally, we provide a comprehensive explanation of the effectiveness of our improved loss function through experiments. Our code is publicly available at https://github.com/Li-fAngyU/sequential_rec.

翻译：近年来，对序列推荐模型进行基准测试以及复现/改进现有模型的研究日益增多。例如，Rendle等人通过调整参数和超参数改进了矩阵分解模型。Petrov和Macdonald开发了一种更高效且更鲁棒的BERT4Rec实现，解决了以往工作中BERT4Rec与SASRec性能对比不一致的问题。值得注意的是，BERT4Rec和SASRec具有相似的网络结构，其主要区别在于训练目标/损失函数。为此，我们分析了序列推荐中常用损失函数的优缺点，并提出了一种融合其优势的改进损失函数。我们在两个具有影响力的开源库上进行了大量实验，结果表明，我们的改进损失函数显著提升了GRU4Rec、SASRec、SR-GNN和S3Rec模型的性能，大幅提高了它们的基准表现。此外，改进后的SASRec基准在ML-1M和Beauty数据集上超越了BERT4Rec，并在ML-20M和Steam数据集上取得了与BERT4Rec相近的结果。我们还复现了BERT4Rec模型在Beauty数据集上的实验结果。最后，通过实验全面解释了改进损失函数的有效性。我们的代码已公开于https://github.com/Li-fAngyU/sequential_rec。