Instruction-finetuned Large Language Models inherit clear political leanings that have been shown to influence downstream task performance. We expand this line of research beyond the two-party system in the US and audit Llama Chat in the context of EU politics in various settings to analyze the model's political knowledge and its ability to reason in context. We adapt, i.e., further fine-tune, Llama Chat on speeches of individual euro-parties from debates in the European Parliament to reevaluate its political leaning based on the EUandI questionnaire. Llama Chat shows considerable knowledge of national parties' positions and is capable of reasoning in context. The adapted, party-specific, models are substantially re-aligned towards respective positions which we see as a starting point for using chat-based LLMs as data-driven conversational engines to assist research in political science.

翻译：指令微调的大语言模型继承了明确的政治倾向，这些倾向已被证明会影响下游任务表现。我们将这一研究方向从美国两党制拓展至欧盟政治背景，并在多种设置下审计Llama Chat的政治知识及其上下文推理能力。我们通过进一步微调，使Llama Chat适配欧洲议会辩论中各欧洲政党的演讲内容，并基于EUandI问卷重新评估其政治倾向。结果显示，Llama Chat对各国家政党的立场具有相当程度的了解，并具备上下文推理能力。经过适配后的政党特定模型显著地向相应立场重新对齐，我们将其视为利用基于聊天的LLM作为数据驱动对话引擎以辅助政治科学研究的起点。