Sequential user modeling, a critical task in personalized recommender systems, focuses on predicting the next item a user would prefer, requiring a deep understanding of user behavior sequences. Despite the remarkable success of Transformer-based models across various domains, their full potential in comprehending user behavior remains untapped. In this paper, we re-examine Transformer-like architectures aiming to advance state-of-the-art performance. We start by revisiting the core building blocks of Transformer-based methods, analyzing the effectiveness of the item-to-item mechanism within the context of sequential user modeling. After conducting a thorough experimental analysis, we identify three essential criteria for devising efficient sequential user models, which we hope will serve as practical guidelines to inspire and shape future designs. Following this, we introduce ConvFormer, a simple but powerful modification to the Transformer architecture that meets these criteria, yielding state-of-the-art results. Additionally, we present an acceleration technique to minimize the complexity associated with processing extremely long sequences. Experiments on four public datasets showcase ConvFormer's superiority and confirm the validity of our proposed criteria.

翻译：序列用户建模是个性化推荐系统中的关键任务，旨在预测用户下一个可能偏好的物品，需要深入理解用户行为序列。尽管基于Transformer的模型在各领域取得了显著成功，但其在理解用户行为方面的全部潜能尚未被充分挖掘。本文重新审视类Transformer架构，旨在推动现有技术水平的进步。我们首先回溯基于Transformer方法的核心构建模块，在序列用户建模背景下分析物品间机制的有效性。通过详尽的实验分析，我们识别出设计高效序列用户模型的三大关键准则——这些准则将作为实用指南，为未来设计提供启示与方向。随后，我们提出ConvFormer——一种满足上述准则的简单而强大的Transformer架构改进，并取得了最先进的性能。此外，我们还提出一种加速技术，以降低处理超长序列时的计算复杂度。在四个公开数据集上的实验展示了ConvFormer的优越性，并验证了所提准则的有效性。