Migration has been a core topic in German political debate, from the postwar displacement of millions of expellees to labor migration and recent refugee movements. Studying political speech across such wide-ranging phenomena in depth has traditionally required extensive manual annotation, limiting analysis to small subsets of the data. Large language models (LLMs) offer a potential way to overcome this constraint. Using a theory-driven annotation scheme, we examine how well LLMs annotate subtypes of solidarity and anti-solidarity in German parliamentary debates and whether the resulting labels support valid downstream inference. We first provide a comprehensive evaluation of multiple LLMs, analyzing the effects of model size, prompting strategies, fine-tuning, historical versus contemporary data, and systematic error patterns. We find that the strongest models, especially GPT-5 and gpt-oss-120B, achieve human-level agreement on this task, although their errors remain systematic and bias downstream results. To address this issue, we combine soft-label model outputs with Design-based Supervised Learning (DSL) to reduce bias in long-term trend estimates. Beyond the methodological evaluation, we interpret the resulting annotations from a social-scientific perspective to trace trends in solidarity and anti-solidarity toward migrants in postwar and contemporary Germany. Our approach shows relatively high levels of solidarity in the postwar period, especially in group-based and compassionate forms, and a marked rise in anti-solidarity since 2015, framed through exclusion, undeservingness, and resource burden. We argue that LLMs can support large-scale social-scientific text analysis, but only when their outputs are rigorously validated and statistically corrected.
翻译:移民一直是德国政治辩论的核心议题,从战后数百万被驱逐者的安置到劳工移民及近期的难民运动。深入分析如此广泛现象中的政治演讲传统上需要大量人工标注,从而将分析局限于数据的子集。大语言模型(LLMs)为克服这一限制提供了潜在途径。基于理论驱动的标注方案,我们考察了LLMs在德国议会辩论中标注团结与反团结子类型的表现,以及所得标签是否支持有效的下游推断。我们首先对多个LLMs进行全面评估,分析模型规模、提示策略、微调、历史数据与当代数据差异及系统错误模式的影响。研究发现,最强的模型(尤其是GPT-5和gpt-oss-120B)在此任务上达到了人类水平的共识,但它们的错误仍具有系统性,并导致下游结果存在偏差。为解决此问题,我们将软标签模型输出与基于设计的监督学习(DSL)相结合,以减少长期趋势估计中的偏差。除方法论层面的评估外,我们从社会科学视角解释所获标注,以追溯战后及当代德国对移民的团结与反团结趋势。我们的方法显示,战后时期团结程度较高,尤其体现在群体性和同情性团结中,而自2015年以来,以排斥、不应得和资源负担为框架的反团结显著上升。我们认为,LLMs可支持大规模社会科学文本分析,但前提是其输出必须经过严格验证和统计校正。