Recent advances in large language models (LLMs) have led to their extensive global deployment, and ensuring their safety calls for comprehensive and multilingual toxicity evaluations. However, existing toxicity benchmarks are overwhelmingly focused on English, posing serious risks to deploying LLMs in other languages. We address this by introducing PolygloToxicityPrompts (PTP), the first large-scale multilingual toxicity evaluation benchmark of 425K naturally occurring prompts spanning 17 languages. We overcome the scarcity of naturally occurring toxicity in web-text and ensure coverage across languages with varying resources by automatically scraping over 100M web-text documents. Using PTP, we investigate research questions to study the impact of model size, prompt language, and instruction and preference-tuning methods on toxicity by benchmarking over 60 LLMs. Notably, we find that toxicity increases as language resources decrease or model size increases. Although instruction- and preference-tuning reduce toxicity, the choice of preference-tuning method does not have any significant impact. Our findings shed light on crucial shortcomings of LLM safeguarding and highlight areas for future research.

翻译：近年来，大型语言模型（LLMs）的进步使其在全球范围内广泛部署，确保其安全性需要进行全面且多语言的毒性评估。然而，现有的毒性基准 overwhelmingly 集中于英语，这给在其他语言中部署LLMs带来了严重风险。为此，我们引入PolygloToxicityPrompts（PTP），这是首个大规模多语言毒性评估基准，包含覆盖17种语言的42.5万个自然发生的提示。我们通过自动抓取超过1亿份网络文本文档，克服了网络文本中自然毒性数据稀缺的问题，并确保了在不同资源水平语言中的覆盖。利用PTP，我们研究了模型规模、提示语言、指令微调和偏好微调方法对毒性的影响，并对超过60个LLM进行了基准测试。值得注意的是，我们发现毒性会随着语言资源减少或模型规模增大而增加。尽管指令微调和偏好微调能降低毒性，但偏好微调方法的选择并未产生显著影响。我们的研究揭示了LLM安全保障中的关键缺陷，并指出了未来研究的方向。