This paper presents LightLM, a lightweight Transformer-based language model for generative recommendation. While Transformer-based generative modeling has gained importance in various AI sub-fields such as NLP and vision, generative recommendation is still in its infancy due to its unique demand on personalized generative modeling. Existing works on generative recommendation often use NLP-oriented Transformer architectures such as T5, GPT, LLaMA and M6, which are heavy-weight and are not specifically designed for recommendation tasks. LightLM tackles the issue by introducing a light-weight deep and narrow Transformer architecture, which is specifically tailored for direct generation of recommendation items. This structure is especially apt for straightforward generative recommendation and stems from the observation that language model does not have to be too wide for this task, as the input predominantly consists of short tokens that are well-suited for the model's capacity. We also show that our devised user and item ID indexing methods, i.e., Spectral Collaborative Indexing (SCI) and Graph Collaborative Indexing (GCI), enables the deep and narrow Transformer architecture to outperform large-scale language models for recommendation. Besides, to address the hallucination problem of generating items as output, we propose the constrained generation process for generative recommenders. Experiments on real-world datasets show that LightLM outperforms various competitive baselines in terms of both recommendation accuracy and efficiency. The code can be found at https://github.com/dongyuanjushi/LightLM.

翻译：本文提出LightLM，一种基于Transformer的轻量级语言模型，用于生成式推荐。尽管基于Transformer的生成式建模在自然语言处理和视觉等人工智能子领域日益重要，但生成式推荐因对个性化生成建模的独特需求仍处于起步阶段。现有生成式推荐研究常采用面向NLP的Transformer架构（如T5、GPT、LLaMA和M6），这些模型规模庞大且非专为推荐任务设计。LightLM通过引入专为直接生成推荐项而设计的轻量级深窄Transformer架构解决该问题。该结构尤其适用于简洁的生成式推荐，其设计源于以下观察：语言模型在此任务中无需过宽，因输入主要由适配模型容量的短令牌构成。我们还表明，所提出的用户与物品ID索引方法——谱协同索引（SCI）和图协同索引（GCI）——使深窄Transformer架构在推荐性能上超越大规模语言模型。此外，为应对生成式输出中的幻觉问题，我们提出生成式推荐器的约束生成流程。在真实数据集上的实验表明，LightLM在推荐准确性与效率两方面均优于多个竞争基线。代码详见https://github.com/dongyuanjushi/LightLM。