Reproducibility of recommender systems research has come under scrutiny during recent years. Along with works focusing on repeating experiments with certain algorithms, the research community has also started discussing various aspects of evaluation and how these affect reproducibility. We add a novel angle to this discussion by examining how unofficial third-party implementations could benefit or hinder reproducibility. Besides giving a general overview, we thoroughly examine six third-party implementations of a popular recommender algorithm and compare them to the official version on five public datasets. In the light of our alarming findings we aim to draw the attention of the research community to this neglected aspect of reproducibility.

翻译：近些年来，推荐系统研究的可复现性备受关注。除了聚焦于使用特定算法重复实验的研究工作外，学术界也开始探讨评估的各个方面及其对可复现性的影响。我们通过考察非官方第三方实现可能如何促进或阻碍可复现性，为这一讨论增添了新的视角。除提供总体概述外，我们深入研究了某流行推荐算法的六个第三方实现，并将它们与官方版本在五个公开数据集上进行了对比。基于这些令人警醒的发现，我们旨在引起研究界对这一被忽视的可复现性问题的关注。