Controversy is a reflection of our zeitgeist, and an important aspect to any discourse. The rise of large language models (LLMs) as conversational systems has increased public reliance on these systems for answers to their various questions. Consequently, it is crucial to systematically examine how these models respond to questions that pertaining to ongoing debates. However, few such datasets exist in providing human-annotated labels reflecting the contemporary discussions. To foster research in this area, we propose a novel construction of a controversial questions dataset, expanding upon the publicly released Quora Question Pairs Dataset. This dataset presents challenges concerning knowledge recency, safety, fairness, and bias. We evaluate different LLMs using a subset of this dataset, illuminating how they handle controversial issues and the stances they adopt. This research ultimately contributes to our understanding of LLMs' interaction with controversial issues, paving the way for improvements in their comprehension and handling of complex societal debates.

翻译：争议性是时代精神的反映，也是任何讨论的重要维度。随着大语言模型作为对话系统的兴起，公众愈发依赖这些系统来解答各类问题。因此，系统性地检验这些模型如何回应涉及持续争议的问题变得至关重要。然而，目前提供反映当代讨论的人工标注标签的数据集尚属罕见。为促进该领域研究，我们提出了一种构建争议性问题数据集的新方法，该数据集基于公开的Quora问答对数据集进行扩展。该数据集在知识时效性、安全性、公平性和偏见方面构成挑战。我们利用该数据集的一个子集评估了不同大语言模型，揭示了它们如何处理争议性问题及所持立场。这项研究最终有助于理解大语言模型与争议性问题的交互方式，为提升其理解与处理复杂社会辩论的能力奠定基础。