Existing approaches to estimating politicians' latent positions along specific dimensions often fail when relevant data is limited. We leverage the embedded knowledge in generative large language models (LLMs) to address this challenge and measure lawmakers' positions along specific political or policy dimensions. We prompt an instruction/dialogue-tuned LLM to pairwise compare lawmakers and then scale the resulting graph using the Bradley-Terry model. We estimate novel measures of U.S. senators' positions on liberal-conservative ideology, gun control, and abortion. Our liberal-conservative scale, used to validate LLM-driven scaling, strongly correlates with existing measures and offsets interpretive gaps, suggesting LLMs synthesize relevant data from internet and digitized media rather than memorizing existing measures. Our gun control and abortion measures -- the first of their kind -- differ from the liberal-conservative scale in face-valid ways and predict interest group ratings and legislator votes better than ideology alone. Our findings suggest LLMs hold promise for solving complex social science measurement problems.
翻译:现有方法在有限数据下估算特定维度上的政治人物潜在立场时往往效果不佳。我们利用生成式大型语言模型(LLMs)内置的知识来解决这一挑战,测量立法者在特定政治或政策维度上的立场。我们提示经过指令/对话微调的LLM对立法者进行两两比较,随后使用Bradley-Terry模型对生成的图进行缩放。我们估算了美国参议员在自由-保守意识形态、枪支管控和堕胎问题上的新立场指标。用于验证LLM驱动缩放的自由-保守尺度与现有测量高度相关,并填补了解释性缺口,这表明LLM综合了互联网和数字化媒体中的相关数据,而非单纯记忆现有测量结果。我们的枪支管控和堕胎指标——此类首创指标——在表面效度上不同于自由-保守尺度,并且比单纯意识形态更能预测利益集团评级和立法者投票行为。研究结果表明,LLM有望解决复杂的社会科学测量问题。