Collaborative information from user-item interactions is a fundamental source of signal in successful recommender systems. Recently, researchers have attempted to incorporate this knowledge into large language model-based recommender approaches (LLMRec) to enhance their performance. However, there has been little fundamental analysis of whether LLMs can effectively reason over collaborative information. In this paper, we analyze the ability of LLMs to reason about collaborative information in recommendation tasks, comparing their performance to traditional matrix factorization (MF) models. We propose a simple and effective method to improve LLMs' reasoning capabilities using retrieval-augmented generation (RAG) over the user-item interaction matrix with four different prompting strategies. Our results show that the LLM outperforms the MF model whenever we provide relevant information in a clear and easy-to-follow format, and prompt the LLM to reason based on it. We observe that with this strategy, in almost all cases, the more information we provide, the better the LLM performs.

翻译：用户-物品交互中的协同信息是成功推荐系统的基础信号来源。近年来，研究人员尝试将这些知识融入基于大语言模型的推荐方法（LLMRec）中以提升其性能。然而，关于大语言模型能否有效推理协同信息的根本性分析仍十分匮乏。本文分析了LLM在推荐任务中推理协同信息的能力，并将其性能与传统矩阵分解（MF）模型进行对比。我们提出一种简单有效的方法，通过基于用户-物品交互矩阵的检索增强生成（RAG）结合四种不同的提示策略来提升LLM的推理能力。实验结果表明，当我们以清晰且易于理解的格式提供相关信息，并促使LLM基于此进行推理时，LLM的性能优于MF模型。我们观察到，采用该策略后，几乎在大多数情况下，提供的信息越多，LLM的表现就越出色。