The success of graph neural network-based models (GNNs) has significantly advanced recommender systems by effectively modeling users and items as a bipartite, undirected graph. However, many original graph-based works often adopt results from baseline papers without verifying their validity for the specific configuration under analysis. Our work addresses this issue by focusing on the replicability of results. We present a code that successfully replicates results from six popular and recent graph recommendation models (NGCF, DGCF, LightGCN, SGL, UltraGCN, and GFCF) on three common benchmark datasets (Gowalla, Yelp 2018, and Amazon Book). Additionally, we compare these graph models with traditional collaborative filtering models that historically performed well in offline evaluations. Furthermore, we extend our study to two new datasets (Allrecipes and BookCrossing) that lack established setups in existing literature. As the performance on these datasets differs from the previous benchmarks, we analyze the impact of specific dataset characteristics on recommendation accuracy. By investigating the information flow from users' neighborhoods, we aim to identify which models are influenced by intrinsic features in the dataset structure. The code to reproduce our experiments is available at: https://github.com/sisinflab/Graph-RSs-Reproducibility.

翻译：图神经网络模型（GNNs）通过将用户和物品有效建模为二分无向图，显著推动了推荐系统的发展。然而，许多基于图的原始工作往往直接采用基线论文中的结果，而未验证其对于所分析特定配置的有效性。我们的研究聚焦于结果的可复现性以解决这一问题。我们提出了一套代码，能够在三个通用基准数据集（Gowalla、Yelp 2018和Amazon Book）上成功复现六种流行且较新的图推荐模型（NGCF、DGCF、LightGCN、SGL、UltraGCN和GFCF）的结果。此外，我们将这些图模型与传统协同过滤模型（这些模型在离线评估中历来表现优异）进行了对比。进一步地，我们将研究扩展到两个缺乏现有文献设定基础的新数据集（Allrecipes和BookCrossing）。由于这些数据集上的性能与之前基准数据集存在差异，我们分析了特定数据集特征对推荐准确性的影响。通过探究来自用户邻域的信息流，我们旨在识别哪些模型会受到数据集结构内在特征的影响。可复现本实验的代码已公开于：https://github.com/sisinflab/Graph-RSs-Reproducibility。