Recent studies increasingly explore Large Language Models (LLMs) as a new paradigm for recommendation systems due to their scalability and world knowledge. However, existing work has three key limitations: (1) most efforts focus on retrieval and ranking, while the reranking phase, critical for refining final recommendations, is largely overlooked; (2) LLMs are typically used in zero-shot or supervised fine-tuning settings, leaving their reasoning abilities, especially those enhanced through reinforcement learning (RL) and high-quality reasoning data, underexploited; (3) items are commonly represented by non-semantic IDs, creating major scalability challenges in industrial systems with billions of identifiers. To address these gaps, we propose the Generative Reasoning Reranker (GR2), an end-to-end framework with a three-stage training pipeline tailored for reranking. First, a pretrained LLM is mid-trained on semantic IDs encoded from non-semantic IDs via a tokenizer achieving $\ge$99% uniqueness. Next, a stronger larger-scale LLM generates high-quality reasoning traces through carefully designed prompting and rejection sampling, which are used for supervised fine-tuning to impart foundational reasoning skills. Finally, we apply Decoupled Clip and Dynamic sAmpling Policy Optimization (DAPO), enabling scalable RL supervision with verifiable rewards designed specifically for reranking. Experiments on two real-world datasets demonstrate GR2's effectiveness: it surpasses the state-of-the-art OneRec-Think by 2.4% in Recall@5 and 1.3% in NDCG@5. Ablations confirm that advanced reasoning traces yield substantial gains across metrics. We further find that RL reward design is crucial in reranking: LLMs tend to exploit reward hacking by preserving item order, motivating conditional verifiable rewards to mitigate this behavior and optimize reranking performance.

翻译：近期研究日益探索将大型语言模型（LLM）作为推荐系统的新范式，因其具备可扩展性和世界知识。然而，现有工作存在三个关键局限：（1）多数研究集中于检索与排序阶段，而对优化最终推荐至关重要的重排序阶段则被严重忽视；（2）LLM通常以零样本或有监督微调方式使用，其通过强化学习（RL）与高质量推理数据增强的推理能力尚未得到充分挖掘；（3）项目普遍以非语义ID表示，这在拥有数十亿标识符的工业系统中引发了严重的可扩展性挑战。为弥补这些不足，我们提出生成式推理重排序器（GR2），这是一个专为重排序设计、包含三阶段训练流程的端到端框架。首先，通过分词器将非语义ID编码为语义ID（唯一性≥99%），并基于此对预训练LLM进行中期训练。随后，采用规模更大的强LLM通过精心设计的提示与拒绝采样生成高质量推理轨迹，用于有监督微调以传授基础推理技能。最后，我们应用解耦裁剪与动态采样策略优化（DAPO），实现具有可验证奖励的、专为重排序设计的可扩展RL监督。在两个真实数据集上的实验证明了GR2的有效性：其在Recall@5指标上超越当前最优方法OneRec-Think 2.4%，在NDCG@5指标上提升1.3%。消融实验证实，高级推理轨迹能带来各项指标的显著增益。我们进一步发现RL奖励设计对重排序至关重要：LLM倾向于通过保持项目顺序进行奖励攻击，这促使我们采用条件可验证奖励以抑制该行为并优化重排序性能。