Large language models (LLMs) open up new horizons for sequential recommendations, owing to their remarkable language comprehension and generation capabilities. However, there are still numerous challenges that should be addressed to successfully implement sequential recommendations empowered by LLMs. Firstly, user behavior patterns are often complex, and relying solely on one-step reasoning from LLMs may lead to incorrect or task-irrelevant responses. Secondly, the prohibitively resource requirements of LLM (e.g., ChatGPT-175B) are overwhelmingly high and impractical for real sequential recommender systems. In this paper, we propose a novel Step-by-step knowLedge dIstillation fraMework for recommendation (SLIM), paving a promising path for sequential recommenders to enjoy the exceptional reasoning capabilities of LLMs in a "slim" (i.e., resource-efficient) manner. We introduce CoT prompting based on user behavior sequences for the larger teacher model. The rationales generated by the teacher model are then utilized as labels to distill the downstream smaller student model (e.g., LLaMA2-7B). In this way, the student model acquires the step-by-step reasoning capabilities in recommendation tasks. We encode the generated rationales from the student model into a dense vector, which empowers recommendation in both ID-based and ID-agnostic scenarios. Extensive experiments demonstrate the effectiveness of SLIM over state-of-the-art baselines, and further analysis showcasing its ability to generate meaningful recommendation reasoning at affordable costs.

翻译：大型语言模型（LLMs）凭借其卓越的语言理解与生成能力，为序列推荐开辟了新前景。然而，要成功实现LLM赋能的序列推荐，仍需应对诸多挑战。首先，用户行为模式往往复杂多样，仅依赖LLM的单步推理可能导致错误或与任务无关的响应。其次，LLM（如ChatGPT-175B）的庞大资源需求过于高昂，在现实序列推荐系统中难以实际应用。本文提出一种新颖的逐步知识蒸馏推荐框架（SLIM），为序列推荐器以“轻量化”（即资源高效）方式享受LLM卓越推理能力铺就了一条可行路径。我们针对大型教师模型引入基于用户行为序列的思维链（CoT）提示。随后，利用教师模型生成的推理过程作为标签，蒸馏下游较小规模的学生模型（如LLaMA2-7B）。通过这种方式，学生模型获得了推荐任务中的逐步推理能力。我们将学生模型生成的推理过程编码为稠密向量，从而在基于标识符（ID-based）和非标识符（ID-agnostic）场景中赋能推荐。大量实验证明了SLIM相较于最先进基线的有效性，进一步分析显示其能以可承受的代价生成有意义的推荐推理。