As large language models (LLMs) are increasingly deployed, understanding how they express political positioning is important for evaluating alignment and downstream effects. We audit 26 contemporary LLMs using three political psychometric inventories (Political Compass, SapplyValues, 8Values) and a news bias labeling task. To test robustness, inventories are administered across multiple semantic prompt variants and analyzed with a two-way ANOVA separating model and prompt effects. Most models cluster in a similar ideological region, with 96.3% located in the Libertarian-Left quadrant of the Political Compass, and model identity explaining most variance across prompt variants ($η^2 > 0.90$). Cross-instrument comparisons suggest that the Political Compass social axis aligns more strongly with cultural progressivism than authority-related measures ($r=-0.64$). We observe differences between open-weight and closed-source models and asymmetric performance in detecting extreme political bias in downstream classification. Regression analysis finds that psychometric ideological positioning does not significantly predict classification errors, providing no evidence of a statistically significant relationship between conversational ideological identity and task-level behavior. These findings suggest that single-axis evaluations are insufficient and that multidimensional auditing frameworks are important to characterize alignment behavior in deployed LLMs. Our code and data are publicly available at https://github.com/sakhadib/PolAlignLLM.
翻译:随着大型语言模型(LLMs)的部署日益广泛,理解其如何表达政治立场对于评估模型对齐性和下游影响至关重要。本研究采用三种政治心理测量量表(政治指南针、SapplyValues、8Values)和新闻偏见标注任务,对26个当代LLMs进行系统性审计。为检验稳健性,各量表通过多种语义提示变体实施,并采用双向方差分析分离模型效应与提示效应。结果显示,大多数模型聚集在相似意识形态区域,其中96.3%位于政治指南针的自由主义-左翼象限,且模型身份解释了提示变体间的主要方差($η^2 > 0.90$)。跨工具比较表明,政治指南针的社会轴与文化进步主义的相关性强于权威相关测量指标($r=-0.64$)。我们观察到开源权重模型与闭源模型之间的差异,以及在下游分类中检测极端政治偏见的不对称表现。回归分析发现,心理测量意识形态定位不能显著预测分类错误,未发现对话式意识形态身份与任务级行为存在统计学显著关系的证据。这些发现表明,单轴评估体系存在不足,需要采用多维审计框架来准确表征已部署LLMs的对齐行为。我们的代码与数据已在https://github.com/sakhadib/PolAlignLLM公开。