Sequential Recommendation (SR) focuses on personalizing user experiences by predicting future preferences based on historical interactions. Transformer models, with their attention mechanisms, have become the dominant architecture in SR tasks due to their ability to capture dependencies in user behavior sequences. However, traditional attention mechanisms, where attention weights are computed through query-key transformations, are inherently linear and deterministic. This fixed approach limits their ability to account for the dynamic and non-linear nature of user preferences, leading to challenges in capturing evolving interests and subtle behavioral patterns. Given that generative models excel at capturing non-linearity and probabilistic variability, we argue that generating attention distributions offers a more flexible and expressive alternative compared to traditional attention mechanisms. To support this claim, we present a theoretical proof demonstrating that generative attention mechanisms offer greater expressiveness and stochasticity than traditional deterministic approaches. Building upon this theoretical foundation, we introduce two generative attention models for SR, each grounded in the principles of Variational Autoencoders (VAE) and Diffusion Models (DMs), respectively. These models are designed specifically to generate adaptive attention distributions that better align with variable user preferences. Extensive experiments on real-world datasets show our models significantly outperform state-of-the-art in both accuracy and diversity.

翻译：序列推荐旨在通过用户历史交互预测其未来偏好，从而实现个性化体验。凭借其注意力机制，Transformer模型已成为序列推荐任务的主流架构，因其能够有效捕捉用户行为序列中的依赖关系。然而，传统注意力机制通过查询-键值变换计算注意力权重，本质上具有线性与确定性特征。这种固定模式限制了其对用户偏好动态性与非线性特征的表达能力，导致在捕捉兴趣演变与细微行为模式时面临挑战。鉴于生成式模型在捕捉非线性与概率可变性方面具有显著优势，我们认为生成注意力分布相较于传统注意力机制能提供更灵活且更具表达力的替代方案。为验证此观点，我们提供了理论证明，表明生成式注意力机制比传统确定性方法具有更强的表达力与随机性。基于此理论框架，我们提出了两种面向序列推荐的生成式注意力模型，分别基于变分自编码器与扩散模型原理构建。这些模型专门设计用于生成自适应注意力分布，以更好地契合动态变化的用户偏好。在真实数据集上的大量实验表明，我们的模型在准确性与多样性方面均显著优于现有最优方法。