This paper presents OpenP5, an open-source library for benchmarking foundation models for recommendation under the Pre-train, Personalized Prompt and Predict Paradigm (P5). We consider the implementation of P5 on three dimensions: 1) downstream task, 2) recommendation dataset, and 3) item indexing method. For 1), we provide implementation over two downstream tasks: sequential recommendation and straightforward recommendation. For 2), we surveyed frequently used datasets in recommender system research in recent years and provide implementation on ten datasets. In particular, we provide both single-dataset implementation and the corresponding checkpoints (P5) and another Super P5 (SP5) implementation that is pre-trained on all of the datasets, which supports recommendation across various domains with one model. For 3), we provide implementation of three item indexing methods to create item IDs: random indexing, sequential indexing, and collaborative indexing. We also provide comprehensive evaluation results of the library over the two downstream tasks, the ten datasets, and the three item indexing methods to facilitate reproducibility and future research. We open-source the code and the pre-trained checkpoints of the OpenP5 library at https://github.com/agiresearch/OpenP5.

翻译：本文提出OpenP5，这是一个在“预训练、个性化提示与预测范式”（P5）下对推荐系统基础模型进行基准测试的开源库。我们从三个维度考虑P5的实现：1）下游任务，2）推荐数据集，以及3）项目索引方法。对于1），我们实现了两个下游任务：序列推荐和直接推荐。对于2），我们调研了近年来推荐系统研究中常用的数据集，并在十个数据集上提供了实现。特别地，我们提供了单一数据集的实现及相应检查点（P5），以及另一个基于所有数据集预训练的超级P5（SP5）实现，该实现支持用一个模型进行跨域推荐。对于3），我们提供了三种项目索引方法的实现：随机索引、顺序索引和协同索引。我们还针对这两个下游任务、十个数据集以及三种项目索引方法提供了库的全面评估结果，以促进可复现性和未来研究。我们在https://github.com/agiresearch/OpenP5上开源了OpenP5库的代码和预训练检查点。