While preliminary findings indicate that multilingual LLMs exhibit reduced bias compared to monolingual ones, a comprehensive understanding of the effect of multilingual training on bias mitigation, is lacking. This study addresses this gap by systematically training six LLMs of identical size (2.6B parameters) and architecture: five monolingual models (English, German, French, Italian, and Spanish) and one multilingual model trained on an equal distribution of data across these languages, all using publicly available data. To ensure robust evaluation, standard bias benchmarks were automatically translated into the five target languages and verified for both translation quality and bias preservation by human annotators. Our results consistently demonstrate that multilingual training effectively mitigates bias. Moreover, we observe that multilingual models achieve not only lower bias but also superior prediction accuracy when compared to monolingual models with the same amount of training data, model architecture, and size.

翻译：尽管初步研究结果表明，多语言大语言模型相比单语言模型展现出更低的偏见，但对于多语言训练在偏见缓解方面的作用，目前仍缺乏全面的理解。本研究通过系统性地训练六个规模相同（26亿参数）且架构一致的模型来填补这一空白：五个单语言模型（英语、德语、法语、意大利语和西班牙语）以及一个多语言模型，该多语言模型使用公开可用的数据，并在这些语言上以等量数据分布进行训练。为确保评估的稳健性，标准的偏见基准测试被自动翻译成五种目标语言，并由人工标注者验证了翻译质量和偏见保持情况。我们的结果一致表明，多语言训练能有效缓解偏见。此外，我们观察到，在训练数据量、模型架构和规模相同的情况下，多语言模型不仅实现了更低的偏见，还获得了优于单语言模型的预测准确率。