Recent studies on pre-trained vision/language models have demonstrated the practical benefit of a new, promising solution-building paradigm in AI where models can be pre-trained on broad data describing a generic task space and then adapted successfully to solve a wide range of downstream tasks, even when training data is severely limited (e.g., in zero- or few-shot learning scenarios). Inspired by such progress, we investigate in this paper the possibilities and challenges of adapting such a paradigm to the context of recommender systems, which is less investigated from the perspective of pre-trained model. In particular, we propose to develop a generic recommender that captures universal interaction patterns by training on generic user-item interaction data extracted from different domains, which can then be fast adapted to improve few-shot learning performance in unseen new domains (with limited data). However, unlike vision/language data which share strong conformity in the semantic space, universal patterns underlying recommendation data collected across different domains (e.g., different countries or different E-commerce platforms) are often occluded by both in-domain and cross-domain biases implicitly imposed by the cultural differences in their user and item bases, as well as their uses of different e-commerce platforms. As shown in our experiments, such heterogeneous biases in the data tend to hinder the effectiveness of the pre-trained model. To address this challenge, we further introduce and formalize a causal debiasing perspective, which is substantiated via a hierarchical Bayesian deep learning model, named PreRec. Our empirical studies on real-world data show that the proposed model could significantly improve the recommendation performance in zero- and few-shot learning settings under both cross-market and cross-platform scenarios.

翻译：近期关于预训练视觉/语言模型的研究表明，一种新的、有前景的AI解决方案构建范式具有实际价值：模型可在描述通用任务空间的广泛数据上进行预训练，随后成功适配解决各类下游任务，即使在训练数据严重受限（如零样本或少样本学习场景）的情况下也不例外。受此类进展启发，本文探讨了将该范式应用于推荐系统领域的可能性与挑战——目前从预训练模型角度对推荐系统的研究尚不充分。具体而言，我们提出通过训练跨不同领域提取的通用用户-物品交互数据，构建一个捕获通用交互模式的通用推荐器，该推荐器可快速适配以改善未知新领域（数据有限）中的少样本学习性能。然而，与语义空间呈现强一致性的视觉/语言数据不同，跨领域（如不同国家或不同电商平台）收集的推荐数据中蕴藏的通用模式，往往受到领域内及跨领域偏差的遮蔽——这些偏差由用户和物品基础的文化差异以及不同电商平台的使用方式所隐性施加。如实验所示，数据中的异质性偏差会阻碍预训练模型的有效性。为应对这一挑战，我们进一步引入并形式化了一种因果去偏视角，并通过层次贝叶斯深度学习模型PreRec加以实现。基于真实数据的实证研究表明，所提模型能够显著提升跨市场与跨平台场景下零样本及少样本学习设置中的推荐性能。