Large Language Models (LLMs) have been shown to contain biases in the process of integrating conflicting information when answering questions. Here we ask whether such biases also exist with respect to which language is used for each conflicting piece of information. To answer this question, we extend the conflicting needles in a haystack paradigm to a multilingual setting and perform a comprehensive set of evaluations with naturalistic news domain data in five different languages, for a range of multilingual LLMs of different sizes. We find that all LLMs tested, including GPT-5.2, ignore the conflict and confidently assert only one of the possible answers in the large majority of cases. Furthermore, there is a consistent bias across models in which languages are preferred, with a general bias against Russian and, for the longest context lengths, in favor of Chinese. Both of these patterns are consistent between models trained inside and outside of mainland China, though somewhat stronger in the former category.
翻译:大型语言模型(LLMs)在整合冲突信息以回答问题的过程中已被证实存在偏差。本研究进一步探究此类偏差是否与每段冲突信息所采用的语言相关。为解答该问题,我们将"干草堆中的冲突针"范式扩展至多语言场景,利用五种不同语言的新闻领域自然语料,对不同规模的多语言LLMs进行了系统性评估。研究发现,包括GPT-5.2在内的所有被测模型在绝大多数情况下均会忽略信息冲突,仅坚定地断言其中一个可能的答案。此外,模型对不同语言存在一致的偏好偏差:普遍对俄语存在偏见,而在最长上下文长度下则偏向中文。上述两种模式在境内外训练的模型中均表现一致,但境内训练模型的偏差程度相对更强。