Characterizing users and items through vector representations is crucial for various tasks in recommender systems. Recent approaches attempt to apply Large Language Models (LLMs) in recommendation through a question and answer format, where real users and items (e.g., Item No.2024) are represented with in-vocabulary tokens (e.g., "item", "20", "24"). However, since LLMs are typically pretrained on natural language tasks, these in-vocabulary tokens lack the expressive power for distinctive users and items, thereby weakening the recommendation ability even after fine-tuning on recommendation tasks. In this paper, we explore how to effectively tokenize users and items in LLM-based recommender systems. We emphasize the role of out-of-vocabulary (OOV) tokens in addition to the in-vocabulary ones and claim the memorization of OOV tokens that capture correlations of users/items as well as diversity of OOV tokens. By clustering the learned representations from historical user-item interactions, we make the representations of user/item combinations share the same OOV tokens if they have similar properties. Furthermore, integrating these OOV tokens into the LLM's vocabulary allows for better distinction between users and items and enhanced capture of user-item relationships during fine-tuning on downstream tasks. Our proposed framework outperforms existing state-of-the-art methods across various downstream recommendation tasks.

翻译：通过向量表示刻画用户与物品特征对于推荐系统中的多项任务至关重要。近期研究尝试通过问答形式将大语言模型应用于推荐任务，其中真实用户与物品（如"Item No.2024"）被表示为词汇内标记（如"item"、"20"、"24"）。然而，由于大语言模型通常在自然语言任务上预训练，这些词汇内标记缺乏对特定用户与物品的表征能力，即使在推荐任务上微调后仍会削弱模型的推荐性能。本文探究了基于大语言模型的推荐系统中如何有效对用户与物品进行标记化处理。我们强调词汇外标记与词汇内标记的协同作用，并提出应同时关注词汇外标记对用户/物品关联性的记忆能力及其多样性。通过对历史用户-物品交互学习到的表征进行聚类，我们使具有相似属性的用户/物品组合共享相同的词汇外标记。此外，将这些词汇外标记集成至大语言模型的词汇表中，能够在下游任务微调过程中更好地区分用户与物品，并增强对用户-物品关系的捕捉能力。实验表明，我们提出的框架在多种下游推荐任务中均优于现有最先进方法。