Large Language Models (LLMs) display remarkable capabilities to understand or even produce political discourse but have been found to consistently exhibit a progressive left-leaning bias. At the same time, so-called persona or identity prompts have been shown to produce LLM behavior that aligns with socioeconomic groups with which the base model is not aligned. In this work, we analyze whether zero-shot persona prompting with limited information can accurately predict individual voting decisions and, by aggregation, accurately predict the positions of European groups on a diverse set of policies. We evaluate whether predictions are stable in response to counterfactual arguments, different persona prompts, and generation methods. Finally, we find that we can simulate the voting behavior of Members of the European Parliament reasonably well, achieving a weighted F1 score of approximately 0.793. Our persona dataset of politicians in the 2024 European Parliament and our code are available at the following url: https://github.com/dess-mannheim/european_parliament_simulation.
翻译:大语言模型(LLMs)展现出理解甚至生成政治话语的卓越能力,但已被发现持续表现出进步左倾的政治倾向。与此同时,所谓的人物角色或身份提示已被证明能够使LLM的行为与基础模型原本未对齐的社会经济群体保持一致。在本研究中,我们分析了在有限信息下使用零样本人物角色提示能否准确预测个体投票决策,并通过聚合预测准确推断欧洲议会各党团在多样化政策议题上的立场。我们评估了预测结果在面对反事实论证、不同人物角色提示及生成方法时的稳定性。最终,我们发现能够较好地模拟欧洲议会议员的投票行为,加权F1分数达到约0.793。我们构建的2024年欧洲议会议员人物角色数据集及相关代码已在以下网址公开:https://github.com/dess-mannheim/european_parliament_simulation。