This paper presents LightLM, a lightweight Transformer-based language model for generative recommendation. While Transformer-based generative modeling has gained importance in various AI sub-fields such as NLP and vision, generative recommendation is still in its infancy due to its unique demand on personalized generative modeling. Existing works on generative recommendation often use NLP-oriented Transformer architectures such as T5, GPT, LLaMA and M6, which are heavy-weight and are not specifically designed for recommendation tasks. LightLM tackles the issue by introducing a light-weight deep and narrow Transformer architecture, which is specifically tailored for direct generation of recommendation items. This structure is especially apt for straightforward generative recommendation and stems from the observation that language model does not have to be too wide for this task, as the input predominantly consists of short tokens that are well-suited for the model's capacity. We also show that our devised user and item ID indexing methods, i.e., Spectral Collaborative Indexing (SCI) and Graph Collaborative Indexing (GCI), enables the deep and narrow Transformer architecture to outperform large-scale language models for recommendation. Besides, to address the hallucination problem of generating items as output, we propose the constrained generation process for generative recommenders. Experiments on real-world datasets show that LightLM outperforms various competitive baselines in terms of both recommendation accuracy and efficiency. The code can be found at https://github.com/dongyuanjushi/LightLM.

翻译：本文提出LightLM，一种基于Transformer的轻量级语言模型，专用于生成式推荐任务。尽管基于Transformer的生成式建模在自然语言处理和视觉等人工智能子领域中愈发重要，但由于生成式推荐对个性化建模的特殊需求，该领域仍处于起步阶段。现有生成式推荐研究通常采用面向自然语言处理的Transformer架构，如T5、GPT、LLaMA和M6，这些模型体量庞大且未针对推荐任务进行专门设计。LightLM通过引入专为直接生成推荐项而设计的轻量级深窄Transformer架构解决该问题。该结构特别适用于直接生成式推荐，其设计源于观察：语言模型在此类任务中无需过宽，因为输入主要由短令牌构成，与模型容量高度适配。我们还展示了所提出的用户与物品ID索引方法（即谱协同索引SCI和图协同索引GCI），能使深窄Transformer架构在推荐性能上超越大规模语言模型。此外，针对生成输出项时的幻觉问题，我们提出了生成式推荐器的约束生成过程。在真实数据集上的实验表明，LightLM在推荐准确性和效率两方面均优于多种强基线模型。代码开源地址：https://github.com/dongyuanjushi/LightLM。