Leveraging users' long engagement histories is essential for personalized content recommendations. The success of pretrained language models (PLMs) in NLP has led to their use in encoding user histories and candidate items, framing content recommendations as textual semantic matching tasks. However, existing works still struggle with processing very long user historical text and insufficient user-item interaction. In this paper, we introduce a content-based recommendation framework, SPAR, which effectively tackles the challenges of holistic user interest extraction from the long user engagement history. It achieves so by leveraging PLM, poly-attention layers and attention sparsity mechanisms to encode user's history in a session-based manner. The user and item side features are sufficiently fused for engagement prediction while maintaining standalone representations for both sides, which is efficient for practical model deployment. Moreover, we enhance user profiling by exploiting large language model (LLM) to extract global interests from user engagement history. Extensive experiments on two benchmark datasets demonstrate that our framework outperforms existing state-of-the-art (SoTA) methods.

翻译：利用用户的长时参与历史对于个性化内容推荐至关重要。预训练语言模型（PLM）在自然语言处理领域的成功，促使其被用于编码用户历史与候选项目，从而将内容推荐构建为文本语义匹配任务。然而，现有方法在处理极长的用户历史文本以及用户-项目交互不足方面仍面临挑战。本文提出一种基于内容的推荐框架SPAR，该框架有效解决了从长用户参与历史中提取整体用户兴趣的难题。其通过利用PLM、多头注意力层及注意力稀疏机制，以会话方式编码用户历史。用户侧与项目侧特征在参与度预测中得到充分融合，同时保持两侧的独立表征，这有利于实际模型部署的效率。此外，我们通过利用大语言模型（LLM）从用户参与历史中提取全局兴趣，从而增强了用户画像构建。在两个基准数据集上的大量实验表明，我们的框架优于现有的最先进方法。