Pretrained Language Models (PLM) have been greatly successful on a board range of natural language processing (NLP) tasks. However, it has just started being applied to the domain of recommendation systems. Traditional recommendation algorithms failed to incorporate the rich textual information in e-commerce datasets, which hinderss the performance of those models. We present a thorough investigation on the effect of various strategy of incorporating PLMs into traditional recommender algorithms on one of the e-commerce datasets, and we compare the results with vanilla recommender baseline models. We show that the application of PLMs and domain specific fine-tuning lead to an increase on the predictive capability of combined models. These results accentuate the importance of utilizing textual information in the context of e-commerce, and provides insight on how to better apply PLMs alongside traditional recommender system algorithms. The code used in this paper is available on Github: https://github.com/NuofanXu/bert_retail_recommender.

翻译：预训练语言模型（PLM）已在广泛的自然语言处理任务中取得巨大成功，但其在推荐系统领域的应用才刚刚起步。传统推荐算法未能充分利用电子商务数据集中丰富的文本信息，这限制了模型的性能。我们系统研究了将预训练语言模型融入传统推荐算法的不同策略在某个电子商务数据集上的效果，并将结果与基础推荐模型进行了比较。研究表明，应用预训练语言模型并进行领域特定微调能够提升组合模型的预测能力。这些结果凸显了在电子商务背景下利用文本信息的重要性，并为如何更好地将预训练语言模型与传统推荐系统算法结合提供了见解。本文所用代码已公开在Github上：https://github.com/NuofanXu/bert_retail_recommender。