Large Language Models (LLMs) increasingly shape public discourse, yet most evaluations of political and economic bias have focused on high-resource, Western languages and contexts. This leaves critical blind spots in low-resource, multilingual regions such as Pakistan, where linguistic identity is closely tied to political, religious, and regional ideologies. We present a systematic evaluation of political bias in 13 state-of-the-art LLMs across five Pakistani languages: Urdu, Punjabi, Sindhi, Pashto, and Balochi. Our framework integrates a culturally adapted Political Compass Test (PCT) with multi-level framing analysis, capturing both ideological stance (economic/social axes) and stylistic framing (content, tone, emphasis). Prompts are aligned with 11 socio-political themes specific to the Pakistani context. Results show that while LLMs predominantly reflect liberal-left orientations consistent with Western training data, they exhibit more authoritarian framing in regional languages, highlighting language-conditioned ideological modulation. We also identify consistent model-specific bias patterns across languages. These findings show the need for culturally grounded, multilingual bias auditing frameworks in global NLP.
翻译:大语言模型日益塑造公共话语,然而大多数关于政治和经济偏见的评估都集中在高资源的西方语言和语境中。这在巴基斯坦等多语言低资源地区留下了关键盲点,这些地区的语言身份与政治、宗教和地区意识形态紧密相连。我们对13个最先进的大语言模型在五种巴基斯坦语言(乌尔都语、旁遮普语、信德语、普什图语和俾路支语)中的政治偏见进行了系统性评估。我们的框架整合了文化适配的政治指南针测试与多层次框架分析,同时捕捉意识形态立场(经济/社会轴)和风格框架(内容、语调、重点)。提示词与巴基斯坦语境下11个特定的社会政治主题保持一致。结果表明,虽然大语言模型主要反映出与西方训练数据一致的自由左翼倾向,但它们在地区语言中表现出更强的威权主义框架,突显了语言条件化的意识形态调制。我们还识别出跨语言一致的模型特定偏见模式。这些发现表明,在全球自然语言处理领域,需要建立基于文化的多语言偏见审计框架。