The expansion of streaming media and e-commerce has led to a boom in recommendation systems, including Sequential recommendation systems, which consider the user's previous interactions with items. In recent years, research has focused on architectural improvements such as transformer blocks and feature extraction that can augment model information. Among these features are context and attributes. Of particular importance is the temporal footprint, which is often considered part of the context and seen in previous publications as interchangeable with positional information. Other publications use positional encodings with little attention to them. In this paper, we analyse positional encodings, showing that they provide relative information between items that are not inferable from the temporal footprint. Furthermore, we evaluate different encodings and how they affect metrics and stability using Amazon datasets. We added some new encodings to help with these problems along the way. We found that we can reach new state-of-the-art results by finding the correct positional encoding, but more importantly, certain encodings stabilise the training.
翻译:流媒体和电子商务的扩张推动了推荐系统的蓬勃发展,包括考虑用户历史物品交互的序列推荐系统。近年来,研究聚焦于架构改进(如Transformer模块)和特征提取,以增强模型信息。其中,特征包括上下文和属性。尤为重要的是时间印记,它常被视作上下文的一部分,并在先前文献中被认为与位置信息可互换。另一些出版物对位置编码的关注甚少。本文分析了位置编码,证明它们提供了物品之间无法从时间印记推断出的相对信息。此外,我们基于亚马逊数据集评估了不同编码对指标和稳定性的影响,并在此过程中引入了几种新编码以缓解相关问题。研究发现,通过选择合适的位置编码,我们能够取得新的最佳结果,但更重要的是,特定编码能稳定训练过程。