With the rise of LLMs, there is an increasing need for intelligent recommendation assistants that can handle complex queries and provide personalized, reasoning-driven recommendations. LLM-based recommenders show potential but face challenges in multi-step reasoning, underscoring the need for reasoning-augmented systems. To address this gap, we propose ReRec, a novel reinforcement fine-tuning (RFT) framework designed to improve LLM reasoning in complex recommendation tasks. Our framework introduces three key components: (1) Dual-Graph Enhanced Reward Shaping, integrating recommendation metrics like NDCG@K with Query Alignment and Preference Alignment Scores to provide fine-grained reward signals for LLM optimization; (2) Reasoning-aware Advantage Estimation, which decomposes LLM outputs into reasoning segments and penalizes incorrect steps to enhance reasoning of recommendation; and (3) Online Curriculum Scheduler, dynamically assess query difficulty and organize training curriculum to ensure stable learning during RFT. Experiments demonstrate that ReRec outperforms state-of-the-art baselines and preserves core abilities like instruction-following and general knowledge. Our codes are available at https://github.com/jiani-huang/ReRec.

翻译：随着大语言模型（LLM）的兴起，对能够处理复杂查询并提供个性化、推理驱动型推荐的智能推荐助手的需求日益增长。基于LLM的推荐系统展现出潜力，但在多步推理方面仍面临挑战，凸显了构建推理增强型系统的必要性。针对这一空白，我们提出ReRec——一种新型强化微调（RFT）框架，旨在提升LLM在复杂推荐任务中的推理能力。该框架包含三个核心组件：（1）双图增强奖励塑造——将NDCG@K等推荐指标与查询对齐分数、偏好对齐分数相融合，为LLM优化提供细粒度奖励信号；（2）推理感知优势估计——将LLM输出分解为推理片段，并对错误步骤施加惩罚以增强推荐推理能力；（3）在线课程调度器——动态评估查询难度并组织训练课程，确保强化微调过程中的稳定学习。实验表明，ReRec在性能上超越现有最优基线，同时保留指令遵循、通用知识等核心能力。相关代码已开源至https://github.com/jiani-huang/ReRec。