Recent sequential recommendation models have combined pre-trained text embeddings of items with item ID embeddings to achieve superior recommendation performance. Despite their effectiveness, the expressive power of text features in these models remains largely unexplored. While most existing models emphasize the importance of ID embeddings in recommendations, our study takes a step further by studying sequential recommendation models that only rely on text features and do not necessitate ID embeddings. Upon examining pretrained text embeddings experimentally, we discover that they reside in an anisotropic semantic space, with an average cosine similarity of over 0.8 between items. We also demonstrate that this anisotropic nature hinders recommendation models from effectively differentiating between item representations and leads to degenerated performance. To address this issue, we propose to employ a pre-processing step known as whitening transformation, which transforms the anisotropic text feature distribution into an isotropic Gaussian distribution. Our experiments show that whitening pre-trained text embeddings in the sequential model can significantly improve recommendation performance. However, the full whitening operation might break the potential manifold of items with similar text semantics. To preserve the original semantics while benefiting from the isotropy of the whitened text features, we introduce WhitenRec+, an ensemble approach that leverages both fully whitened and relaxed whitened item representations for effective recommendations. We further discuss and analyze the benefits of our design through experiments and proofs. Experimental results on three public benchmark datasets demonstrate that WhitenRec+ outperforms state-of-the-art methods for sequential recommendation.

翻译：近期序列推荐模型将项目的预训练文本嵌入与项目ID嵌入相结合，以取得卓越的推荐性能。尽管效果显著，这些模型中文本特征的表达能力仍未得到充分探索。虽然现有模型大多强调ID嵌入在推荐中的重要性，我们的研究进一步探索了仅依赖文本特征且无需ID嵌入的序列推荐模型。通过实验检验预训练文本嵌入，我们发现它们位于各向异性的语义空间中，项目间的平均余弦相似度超过0.8。我们还证明这种各向异性特性阻碍了推荐模型有效区分项目表示，并导致性能下降。为解决此问题，我们提出采用白化变换作为预处理步骤，将各向异性的文本特征分布转化为各向同性的高斯分布。实验表明，在序列模型中对预训练文本嵌入进行白化处理能显著提升推荐性能。然而，完全白化操作可能破坏具有相似文本语义的项目潜在流形。为在保留原始语义的同时受益于白化文本特征的各向同性，我们引入WhitenRec+——一种集成方法，同时利用完全白化和松弛白化的项目表示实现高效推荐。我们通过实验和理论证明进一步讨论并分析了设计优势。在三个公开基准数据集上的实验结果表明，WhitenRec+在序列推荐任务中优于现有最先进方法。