Generative models have recently demonstrated strong potential in multi-behavior recommendation systems, leveraging the expressive power of transformers and tokenization to generate personalized item sequences. However, their adoption is hindered by (1) the lack of explicit information for token reasoning, (2) high computational costs due to quadratic attention complexity and dense sequence representations after tokenization, and (3) limited multi-scale modeling over user history. In this work, we propose GRACE (Generative Recommendation via journey-aware sparse Attention on Chain-of-thought tokEnization), a novel generative framework for multi-behavior sequential recommendation. GRACE introduces a hybrid Chain-of-Thought (CoT) tokenization method that encodes user-item interactions with explicit attributes from product knowledge graphs (e.g., category, brand, price) over semantic tokenization, enabling interpretable and behavior-aligned generation. To address the inefficiency of standard attention, we design a Journey-Aware Sparse Attention (JSA) mechanism, which selectively attends to compressed, intra-, inter-, and current-context segments in the tokenized sequence. Experiments on two real-world datasets show that GRACE significantly outperforms state-of-the-art baselines, achieving up to +106.9% HR@10 and +106.7% NDCG@10 improvement over the state-of-the-art baseline on the Home domain, and +22.1% HR@10 on the Electronics domain. GRACE also reduces attention computation by up to 48% with long sequences.

翻译：生成式模型近期在多行为推荐系统中展现出强大潜力，其利用Transformer的表达能力和标记化技术来生成个性化物品序列。然而，其应用受到以下因素制约：(1) 缺乏用于标记推理的显式信息；(2) 二次注意力复杂度及标记化后稠密序列表示导致的高计算成本；(3) 对用户历史的多尺度建模能力有限。本文提出GRACE（基于思维链标记化的旅程感知稀疏注意力生成式推荐），一种用于多行为序列推荐的新型生成式框架。GRACE引入一种混合思维链（CoT）标记化方法，该方法在语义标记化基础上，结合产品知识图谱中的显式属性（如类别、品牌、价格）对用户-物品交互进行编码，从而实现可解释且行为对齐的生成。为解决标准注意力的低效问题，我们设计了一种旅程感知稀疏注意力（JSA）机制，该机制选择性地关注标记化序列中经过压缩的片段、内部上下文片段、跨上下文片段及当前上下文片段。在两个真实数据集上的实验表明，GRACE显著优于当前最先进的基线模型，在Home领域实现了相比最优基线高达+106.9%的HR@10和+106.7%的NDCG@10提升，在Electronics领域实现了+22.1%的HR@10提升。GRACE在长序列场景下还能将注意力计算量降低最高达48%。