Much of social science is centered around terms like ``ideology'' or ``power'', which generally elude precise definition, and whose contextual meanings are trapped in surrounding language. This paper explores the use of large language models (LLMs) to flexibly navigate the conceptual clutter inherent to social scientific measurement tasks. We rely on LLMs' remarkable linguistic fluency to elicit ideological scales of both legislators and text, which accord closely to established methods and our own judgement. A key aspect of our approach is that we elicit such scores directly, instructing the LLM to furnish numeric scores itself. This approach affords a great deal of flexibility, which we showcase through a variety of different case studies. Our results suggest that LLMs can be used to characterize highly subtle and diffuse manifestations of political ideology in text.
翻译:社会科学中的许多核心概念,如"意识形态"或"权力",通常难以精确定义,且其语境含义深嵌于相关语言之中。本文探讨了如何利用大语言模型(LLMs)灵活应对社会科学测量任务中固有的概念混杂问题。我们借助LLMs卓越的语言流畅性,提取出符合既有方法与主观判断的立法者与文本意识形态尺度。本方法的关键在于直接引导LLM生成数值化评分,从而获得极大的灵活性——我们通过多个案例研究展示了这一特性。结果表明,LLMs能够有效表征文本中高度微妙且弥散的政治意识形态表现形式。