As large language models (LLMs) are increasingly integrated into social decision-making, understanding their political positioning and alignment behavior is critical for safety and fairness. This study presents a sociotechnical audit of 26 prominent LLMs, triangulating their positions across three psychometric inventories (Political Compass, SapplyValues, 8 Values) and evaluating their performance on a large-scale news labeling task ($N \approx 27{,}000$). Our results reveal a strong clustering of models in the Libertarian-Left region of the ideological space, encompassing 96.3% of the cohort. Alignment signals appear to be consistent architectural traits rather than stochastic noise ($η^2 > 0.90$); however, we identify substantial discrepancies in measurement validity. In particular, the Political Compass exhibits a strong negative correlation with cultural progressivism ($r=-0.64$) when compared against multi-axial instruments, suggesting a conflation of social conservatism with authoritarianism in this context. We further observe a significant divergence between open-weights and closed-source models, with the latter displaying markedly higher cultural progressivism scores ($p<10^{-25}$). In downstream media analysis, models exhibit a systematic "center-shift," frequently categorizing neutral articles as left-leaning, alongside an asymmetric detection capability in which "Far Left" content is identified with greater accuracy (19.2%) than "Far Right" content (2.0%). These findings suggest that single-axis evaluations are insufficient and that multidimensional auditing frameworks are necessary to characterize alignment behavior in deployed LLMs. Our code and data will be made public.
翻译:随着大语言模型(LLMs)日益融入社会决策过程,理解其政治立场与倾向性行为对安全性与公平性至关重要。本研究对26个主流LLMs进行了社会技术审计,通过三项心理测量量表(政治指南针、SapplyValues、8 Values)交叉验证其立场,并评估其在大规模新闻标注任务($N \approx 27{,}000$)中的表现。结果显示,模型在意识形态空间的自由左翼区域呈现显著聚集现象,覆盖了96.3%的样本。倾向性信号表现为稳定的架构特征而非随机噪声($η^2 > 0.90$),但我们发现测量效度存在显著差异。具体而言,与多轴测量工具相比,政治指南针量表显示出与文化进步主义存在强烈负相关($r=-0.64$),表明在该语境下社会保守主义与威权主义概念被混淆。我们进一步观察到开源权重模型与闭源模型之间存在显著分歧,后者表现出明显更高的文化进步主义得分($p<10^{-25}$)。在下游媒体分析中,模型呈现系统性“中心偏移”现象,频繁将中立文章归类为左倾内容,同时表现出非对称检测能力——对“极左”内容的识别准确率(19.2%)显著高于“极右”内容(2.0%)。这些发现表明,单轴评估框架存在不足,需要采用多维审计框架来准确表征已部署LLMs的倾向性行为。我们的代码与数据将公开提供。