Online news platforms commonly employ personalized news recommendation methods to assist users in discovering interesting articles, and many previous works have utilized language model techniques to capture user interests and understand news content. With the emergence of large language models like GPT-3 and T-5, a new recommendation paradigm has emerged, leveraging pre-trained language models for making recommendations. ChatGPT, with its user-friendly interface and growing popularity, has become a prominent choice for text-based tasks. Considering the growing reliance on ChatGPT for language tasks, the importance of news recommendation in addressing social issues, and the trend of using language models in recommendations, this study conducts an initial investigation of ChatGPT's performance in news recommendations, focusing on three perspectives: personalized news recommendation, news provider fairness, and fake news detection. ChatGPT has the limitation that its output is sensitive to the input phrasing. We therefore aim to explore the constraints present in the generated responses of ChatGPT for each perspective. Additionally, we investigate whether specific prompt formats can alleviate these constraints or if these limitations require further attention from researchers in the future. We also surpass fixed evaluations by developing a webpage to monitor ChatGPT's performance on weekly basis on the tasks and prompts we investigated. Our aim is to contribute to and encourage more researchers to engage in the study of enhancing news recommendation performance through the utilization of large language models such as ChatGPT.
翻译:在线新闻平台普遍采用个性化新闻推荐方法帮助用户发现感兴趣的文章,以往许多工作利用语言模型技术捕捉用户兴趣并理解新闻内容。随着GPT-3和T-5等大型语言模型的出现,一种利用预训练语言模型进行推荐的新范式应运而生。ChatGPT凭借其用户友好的界面和日益普及的特性,已成为文本任务的主流选择。考虑到日益依赖ChatGPT处理语言任务、新闻推荐在解决社会问题中的重要性,以及使用语言模型进行推荐的趋势,本研究从三个角度初步探究了ChatGPT在新闻推荐中的表现:个性化新闻推荐、新闻提供者公平性以及假新闻检测。ChatGPT的一个局限性在于其输出对输入措辞敏感,因此我们旨在探索ChatGPT针对每个角度生成回复时存在的约束。此外,我们研究特定提示格式是否能缓解这些约束,或者这些限制是否需要研究人员未来进一步关注。我们还超越固定评估方式,开发了一个网页,以每周为单位监控ChatGPT在我们所研究的任务和提示上的表现。我们的目标是为利用ChatGPT等大型语言模型提升新闻推荐性能的研究做出贡献,并鼓励更多研究者参与其中。