Instruction-finetuned Large Language Models inherit clear political leanings that have been shown to influence downstream task performance. We expand this line of research beyond the two-party system in the US and audit Llama Chat in the context of EU politics in various settings to analyze the model's political knowledge and its ability to reason in context. We adapt, i.e., further fine-tune, Llama Chat on speeches of individual euro-parties from debates in the European Parliament to reevaluate its political leaning based on the EUandI questionnaire. Llama Chat shows considerable knowledge of national parties' positions and is capable of reasoning in context. The adapted, party-specific, models are substantially re-aligned towards respective positions which we see as a starting point for using chat-based LLMs as data-driven conversational engines to assist research in political science.

翻译：指令微调后的大语言模型继承了显著的政治倾向，这种倾向已被证明会影响下游任务表现。我们将此类研究拓展至美国两党制之外的领域，通过多种设置对欧盟政治语境下的Llama Chat进行审计，分析模型的政治知识及其情境推理能力。我们基于欧洲议会辩论中个体欧洲政党的演讲，对Llama Chat进行适配（即进一步微调），并利用EUandI问卷重新评估其政治倾向。Llama Chat展现出对各国政党立场的充分认知，并具备情境推理能力。经适配后的各政党专属模型显著地向对应政党立场重新对齐，我们认为这为将基于聊天的LLM作为数据驱动型对话引擎，以辅助政治科学研究提供了起点。