Recent advances in generative artificial intelligence, particularly large language models (LLMs), have opened new opportunities for enhancing recommender systems (RecSys). Most existing LLM-based RecSys approaches operate in a discrete space, using vector-quantized tokenizers to align with the inherent discrete nature of language models. However, these quantization methods often result in lossy tokenization and suboptimal learning, primarily due to inaccurate gradient propagation caused by the non-differentiable argmin operation in standard vector quantization. Inspired by the emerging trend of embracing continuous tokens in language models, we propose ContRec, a novel framework that seamlessly integrates continuous tokens into LLM-based RecSys. Specifically, ContRec consists of two key modules: a sigma-VAE Tokenizer, which encodes users/items with continuous tokens; and a Dispersive Diffusion module, which captures implicit user preference. The tokenizer is trained with a continuous Variational Auto-Encoder (VAE) objective, where three effective techniques are adopted to avoid representation collapse. By conditioning on the previously generated tokens of the LLM backbone during user modeling, the Dispersive Diffusion module performs a conditional diffusion process with a novel Dispersive Loss, enabling high-quality user preference generation through next-token diffusion. Finally, ContRec leverages both the textual reasoning output from the LLM and the latent representations produced by the diffusion model for Top-K item retrieval, thereby delivering comprehensive recommendation results. Extensive experiments on four datasets demonstrate that \ourname{} consistently outperforms both traditional and SOTA LLM-based recommender systems. Our results highlight the potential of continuous tokenization and generative modeling for advancing the next generation of recommender systems.

翻译：近年来，生成式人工智能（尤其是大语言模型）的进展为推荐系统的增强开辟了新的机遇。现有的大多数基于大语言模型的推荐系统方法在离散空间中运行，利用向量量化分词器以适配语言模型固有的离散特性。然而，这些量化方法通常会导致有损的分词处理和次优的学习效果，主要原因是标准向量量化中不可微的 argmin 操作导致了梯度传播不准确。受语言模型中采用连续令牌这一新兴趋势的启发，我们提出了 ContRec，一个将连续令牌无缝集成到基于大语言模型的推荐系统中的新颖框架。具体而言，ContRec 包含两个关键模块：一个 sigma-VAE 分词器，用于以连续令牌编码用户/物品；以及一个分散扩散模块，用于捕捉隐式的用户偏好。该分词器通过连续的变分自编码器目标进行训练，并采用了三种有效技术来避免表示坍缩。在用户建模过程中，通过以大语言模型主干先前生成的令牌为条件，分散扩散模块执行一种带有新颖分散损失的条件扩散过程，从而通过下一令牌扩散实现高质量的用户偏好生成。最后，ContRec 同时利用大语言模型的文本推理输出和扩散模型产生的潜在表示进行 Top-K 物品检索，从而提供全面的推荐结果。在四个数据集上进行的大量实验表明，\ourname{} 在性能上持续优于传统的以及最先进的基于大语言模型的推荐系统。我们的结果凸显了连续分词和生成式建模在推动下一代推荐系统发展方面的潜力。