Sequential user modeling, a critical task in personalized recommender systems, focuses on predicting the next item a user would prefer, requiring a deep understanding of user behavior sequences. Despite the remarkable success of Transformer-based models across various domains, their full potential in comprehending user behavior remains untapped. In this paper, we re-examine Transformer-like architectures aiming to advance state-of-the-art performance. We start by revisiting the core building blocks of Transformer-based methods, analyzing the effectiveness of the item-to-item mechanism within the context of sequential user modeling. After conducting a thorough experimental analysis, we identify three essential criteria for devising efficient sequential user models, which we hope will serve as practical guidelines to inspire and shape future designs. Following this, we introduce ConvFormer, a simple but powerful modification to the Transformer architecture that meets these criteria, yielding state-of-the-art results. Additionally, we present an acceleration technique to minimize the complexity associated with processing extremely long sequences. Experiments on four public datasets showcase ConvFormer's superiority and confirm the validity of our proposed criteria.

翻译：序列用户建模是个性化推荐系统中的关键任务，其核心在于预测用户可能偏好的下一项内容，需要深度理解用户行为序列。尽管基于Transformer的模型在各领域取得了显著成功，但其在理解用户行为方面的潜力尚未被充分挖掘。本文旨在重新审视类Transformer架构以推动性能前沿的突破。我们首先回顾Transformer方法的核心构建模块，在序列用户建模背景下分析"项目-项目"机制的有效性。通过详尽的实验分析，我们识别出设计高效序列用户模型的三个关键准则，这些准则有望作为实践指南启发并塑造未来设计。在此基础上，我们提出ConvFormer——一种满足这些准则的简单而强大的Transformer架构改进方案，实现了领先的性能结果。此外，我们引入了一种加速技术以降低处理极长序列的复杂度。在四个公开数据集上的实验表明ConvFormer具有优越性，同时验证了我们提出的准则的有效性。