Public opinion surveys show cross-cultural differences in political opinions between socio-cultural contexts. However, there is no clear evidence whether these differences translate to cross-lingual differences in multilingual large language models (MLLMs). We analyze whether opinions transfer between languages or whether there are separate opinions for each language in MLLMs of various sizes across five Western languages. We evaluate MLLMs' opinions by prompting them to report their (dis)agreement with political statements from voting advice applications. To better understand the interaction between languages in the models, we evaluate them both before and after aligning them with more left or right views using direct preference optimization and English alignment data only. Our findings reveal that unaligned models show only very few significant cross-lingual differences in the political opinions they reflect. The political alignment shifts opinions almost uniformly across all five languages. We conclude that in Western language contexts, political opinions transfer between languages, demonstrating the challenges in achieving explicit socio-linguistic, cultural, and political alignment of MLLMs.
翻译:民意调查显示,不同社会文化背景下的政治观点存在跨文化差异。然而,目前尚无明确证据表明这些差异是否转化为多语言大语言模型(MLLMs)中的跨语言差异。我们分析了五种西方语言中不同规模的MLLMs,探究政治观点是否在语言间迁移,抑或每种语言存在独立的观点体系。我们通过提示MLLMs报告其对投票建议应用中政治陈述的(不)赞同程度来评估其观点。为更好地理解模型中语言间的相互作用,我们评估了模型在使用直接偏好优化和仅英语对齐数据将其与更左翼或右翼观点对齐前后的表现。研究结果表明,未对齐模型反映的政治观点仅存在极少数显著的跨语言差异。政治对齐过程几乎一致地改变了所有五种语言中的观点。我们得出结论:在西方语言语境中,政治观点会在语言间迁移,这凸显了实现MLLMs明确的社会语言学、文化和政治对齐所面临的挑战。