Controversy is a reflection of our zeitgeist, and an important aspect to any discourse. The rise of large language models (LLMs) as conversational systems has increased public reliance on these systems for answers to their various questions. Consequently, it is crucial to systematically examine how these models respond to questions that pertaining to ongoing debates. However, few such datasets exist in providing human-annotated labels reflecting the contemporary discussions. To foster research in this area, we propose a novel construction of a controversial questions dataset, expanding upon the publicly released Quora Question Pairs Dataset. This dataset presents challenges concerning knowledge recency, safety, fairness, and bias. We evaluate different LLMs using a subset of this dataset, illuminating how they handle controversial issues and the stances they adopt. This research ultimately contributes to our understanding of LLMs' interaction with controversial issues, paving the way for improvements in their comprehension and handling of complex societal debates.

翻译：争议性是时代精神的反映，也是任何 discourse 中的重要维度。随着大语言模型作为对话系统的兴起，公众愈发依赖这些系统回答各类问题。因此，系统性地考察这些模型如何回应涉及持续辩论的问题至关重要。然而，目前鲜有数据集能提供反映当代讨论的人工标注标签。为促进该领域研究，我们提出了一种新颖的争议性问题数据集构建方法，在公开的 Quora 问题对数据集基础上进行扩展。该数据集在知识时效性、安全性、公平性和偏见方面均构成挑战。我们利用该数据集的子集评估不同大语言模型，揭示了它们如何处理争议性问题以及所采取的立场。本研究最终有助于理解大语言模型与争议性议题的交互方式，为提升其对复杂社会辩论的理解与处理能力铺平道路。