Leveraging users' long engagement histories is essential for personalized content recommendations. The success of pretrained language models (PLMs) in NLP has led to their use in encoding user histories and candidate items, framing content recommendations as textual semantic matching tasks. However, existing works still struggle with processing very long user historical text and insufficient user-item interaction. In this paper, we introduce a content-based recommendation framework, SPAR, which effectively tackles the challenges of holistic user interest extraction from the long user engagement history. It achieves so by leveraging PLM, poly-attention layers and attention sparsity mechanisms to encode user's history in a session-based manner. The user and item side features are sufficiently fused for engagement prediction while maintaining standalone representations for both sides, which is efficient for practical model deployment. Moreover, we enhance user profiling by exploiting large language model (LLM) to extract global interests from user engagement history. Extensive experiments on two benchmark datasets demonstrate that our framework outperforms existing state-of-the-art (SoTA) methods.

翻译：利用用户长期交互历史是实现个性化内容推荐的关键。自然语言处理中预训练语言模型（PLM）的成功，推动了将其应用于编码用户历史与候选项目，并将内容推荐建模为文本语义匹配任务。然而，现有方法在处理超长用户历史文本及用户-项目交互稀疏性方面仍存在不足。本文提出一种基于内容的推荐框架SPAR，有效解决了从长程用户交互历史中提取整体兴趣的挑战。该框架通过结合PLM、多注意力层（poly-attention layers）与注意力稀疏机制，以会话式方式编码用户历史。用户侧与项目侧特征在保持独立表征的同时充分融合以预测交互行为，这有利于实际模型部署。此外，我们利用大型语言模型（LLM）从用户交互历史中提取全局兴趣，以增强用户画像构建。在两个基准数据集上的广泛实验表明，我们的框架优于现有最先进（SoTA）方法。