With the significant successes of large language models (LLMs) in many natural language processing tasks, there is growing interest among researchers in exploring LLMs for novel recommender systems. However, we have observed that directly using LLMs as a recommender system is usually unstable due to its inherent position bias. To this end, we introduce exploratory research and find consistent patterns of positional bias in LLMs that influence the performance of recommendation across a range of scenarios. Then, we propose a Bayesian probabilistic framework, STELLA (Stable LLM for Recommendation), which involves a two-stage pipeline. During the first probing stage, we identify patterns in a transition matrix using a probing detection dataset. And in the second recommendation stage, a Bayesian strategy is employed to adjust the biased output of LLMs with an entropy indicator. Therefore, our framework can capitalize on existing pattern information to calibrate instability of LLMs, and enhance recommendation performance. Finally, extensive experiments clearly validate the effectiveness of our framework.

翻译：随着大型语言模型（LLMs）在众多自然语言处理任务中取得显著成功，研究人员对探索将LLMs用于新型推荐系统的兴趣日益增长。然而，我们观察到，由于LLMs固有的位置偏差，直接将其用作推荐系统通常是不稳定的。为此，我们开展探索性研究，发现LLMs中位置偏差的一致模式会影响多种场景下的推荐性能。随后，我们提出一个贝叶斯概率框架——STELLA（用于推荐的稳定大语言模型），该框架包含两阶段流程。在第一探测阶段，我们利用探测检测数据集识别转移矩阵中的模式。在第二推荐阶段，采用贝叶斯策略，借助熵指标调整LLMs的有偏输出。因此，我们的框架能够利用现有模式信息来校准LLMs的不稳定性，并提升推荐性能。最后，大量实验清晰验证了我们框架的有效性。