Explainable recommendation has attracted much attention from the industry and academic communities. It has shown great potential for improving the recommendation persuasiveness, informativeness and user satisfaction. Despite a lot of promising explainable recommender models have been proposed in the past few years, the evaluation strategies of these models suffer from several limitations. For example, the explanation ground truths are not labeled by real users, the explanations are mostly evaluated based on only one aspect and the evaluation strategies can be hard to unify. To alleviate the above problems, we propose to build an explainable recommendation dataset with multi-aspect real user labeled ground truths. In specific, we firstly develop a video recommendation platform, where a series of questions around the recommendation explainability are carefully designed. Then, we recruit about 3000 users with different backgrounds to use the system, and collect their behaviors and feedback to our questions. In this paper, we detail the construction process of our dataset and also provide extensive analysis on its characteristics. In addition, we develop a library, where ten well-known explainable recommender models are implemented in a unified framework. Based on this library, we build several benchmarks for different explainable recommendation tasks. At last, we present many new opportunities brought by our dataset, which are expected to shed some new lights to the explainable recommendation field. Our dataset, library and the related documents have been released at https://reasoner2023.github.io/.

翻译：可解释推荐已引起工业界和学术界的广泛关注，其在提升推荐说服力、信息量和用户满意度方面展现出巨大潜力。尽管过去几年中提出了许多有前景的可解释推荐模型，但这些模型的评估策略存在若干局限。例如，解释真值并非由真实用户标注，解释通常仅基于单一维度进行评估，且评估策略难以统一。为缓解上述问题，我们提出构建一个包含多方面真实用户标注真值的可解释推荐数据集。具体而言，我们首先开发了一个视频推荐平台，并围绕推荐可解释性精心设计了一系列问题。随后，我们招募了约3000名不同背景的用户使用该系统，收集其行为数据及对问题的反馈。本文详细阐述了数据集的构建过程，并对其特征进行了广泛分析。此外，我们开发了一个统一框架下的算法库，其中实现了十个知名可解释推荐模型。基于该库，我们针对不同可解释推荐任务构建了多个基准测试。最后，我们展示了数据集带来的诸多新机遇，有望为可解释推荐领域开辟新思路。我们的数据集、算法库及相关文档已发布于https://reasoner2023.github.io/。