Adapting Large Language Models (LLMs) for recommendation requires careful consideration of the decoding process, given the inherent differences between generating items and natural language. Existing approaches often directly apply LLMs' original decoding methods. However, we find these methods encounter significant challenges: 1) amplification bias -- where standard length normalization inflates scores for items containing tokens with generation probabilities close to 1 (termed ghost tokens), and 2) homogeneity issue -- generating multiple similar or repetitive items for a user. To tackle these challenges, we introduce a new decoding approach named Debiasing-Diversifying Decoding (D3). D3 disables length normalization for ghost tokens to alleviate amplification bias, and it incorporates a text-free assistant model to encourage tokens less frequently generated by LLMs for counteracting recommendation homogeneity. Extensive experiments on real-world datasets demonstrate the method's effectiveness in enhancing accuracy and diversity.
翻译:将大语言模型(LLMs)适配于推荐任务时,鉴于生成物品与生成自然语言之间的固有差异,需要仔细考量解码过程。现有方法通常直接应用LLMs原有的解码方法。然而,我们发现这些方法面临显著挑战:1)放大偏差——标准长度归一化会过度放大那些包含生成概率接近1的令牌(称为幽灵令牌)的物品的分数;2)同质性问题——为用户生成多个相似或重复的物品。为应对这些挑战,我们提出了一种名为去偏-多样化解码(D3)的新解码方法。D3通过禁用对幽灵令牌的长度归一化来缓解放大偏差,并引入一个无文本的辅助模型,以鼓励LLMs生成频率较低的令牌,从而对抗推荐同质性。在真实世界数据集上进行的大量实验证明了该方法在提升准确性和多样性方面的有效性。